{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Report1 - 随机森林算法预测交通事故理赔模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**姓名：刘长奇**  \n",
    "**学号：2019300677**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "@[TOC]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 任务简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;交通事故理赔审核预测问题属于二分类问题，问题的实质是根据给定的数据集进行机器学习或神经网络训练，得到相应规律后用\n",
    "来预测新的事故情况是否符合理赔标准，从而辅助保险公司准确判断相应事故是否应该给予理赔。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;通过观察数据集，我们知道理赔员到现场需要确定**36**项数据后才能给出保险公司是否应该给出理赔的结论。至此，问题转化\n",
    "成对于输入一个**36**维的数据，能否建立起输入与输出之间的关系，从而实现对于任意一种交通事故，我们将数据输入计算机，从而让计算机帮助我们快速判断该交通事故保险公司是否应该理赔。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题的求解思路"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;通过对带标签的数据进行观察，首先发现数据集的一大特点是：数据量大，输入信息多而杂；由此带来的**好处是构建的学习算法能够对此类问题有更好的针对性**，更贴近真实情况，能够让最终得出的学习模型有更好的实用价值；但由此带来的**坏处是不能直观判断出数据是否存在异常**，因此，在进行分类任务前，对数据的清洗处理是不可或缺的。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;经过数据的预处理后，对于多输入的分类模型，首先想到的是**神经网络**算法，通过构建适当的神经网络，让训练集的预测准确率达到较高的水准后，再用已得到的神经网络模型对预测数据进行预测，从而实现模型的初步建立。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;**随机森林分类模型**也是一种常用的分类模型，它和神经网络类似，均可以对较多维度的数据输入进行分类预测，本文主要通过对随机森林分类模型和神经网络预测模型进行对比，借助**混淆矩阵**理论和**AUC模型评估**方法，对两种模型的优劣做出评价，从而得到更适合用于交通事故理赔预测问题的模型。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;针对模型的改进，本人的思路来源于对数据的直观观察。在观察数据时发现：**虽然输入的数据维数很高，但大多数都是0**，因此，本人通过数据降维的方法对数据信息进行提取，发现在数据维度大大降低之后，原数据的信息量并没有过多丢失。将降维处理后的数据应用到原预测模型中，在经过综合的比较和判断，最终确定，**无需降维的随机森林分类器**效果最佳。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 神经网络分类（未降维处理）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;神经网络是一种通用的分类器，它优势在于可以在各种不同的数据之间寻找规律，从而构建起输入与输出之间的关系，达到良好的分类效果。\n",
    "\n",
    "### 多层神经网络原理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![ ](https://gitee.com/pi-lab/machinelearning_notebook/raw/master/5_nn/images/nn1.jpeg)\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;神经网络其实就是按照一定规则连接起来的多个神经元。上图展示了一个全连接(full connected, FC)神经网络，通过观察上面的图，我们可以发现它的规则包括：\n",
    "\n",
    "**神经元按照层来布局:**<br>\n",
    "&ensp;&ensp;&ensp;&ensp;(1)、最左边的层叫做输入层，负责接收输入数据；<br>\n",
    "&ensp;&ensp;&ensp;&ensp;(2)、最右边的层叫输出层，我们可以从这层获取神经网络输出数据;<br>\n",
    "&ensp;&ensp;&ensp;&ensp;(3)、输入层和输出层之间的层叫做隐藏层，因为它们对于外部来说是不可见的。\n",
    "\n",
    "**同一层的神经元之间没有连接。**<br>\n",
    "**第N层的每个神经元和第N-1层的所有神经元相连(这就是full connected的含义)，第N-1层神经元的输出就是第N层神经元的输入。**<br>\n",
    "**每个连接都有一个权值。**<br>\n",
    "\n",
    "### 神经网络的输出<br>\n",
    "&ensp;&ensp;&ensp;&ensp;神经网络实际上就是一个输入向量$\\vec{x}$到输出向量$\\vec{y}$的函数，即:$$ \\vec{y}=f_network(\\vec{x}) $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;神经网络的输出采用$sigmoid$函数，公式如下：$$sigmoid(x)=\\frac{1}{1+e^{-x}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;而我们要做的工作就是完成**前向计算**和**后向计算**两个过程，直至达到所需精度<br>\n",
    "&ensp;&ensp;&ensp;&ensp;前向计算：按照神经网络示意图将参数向前计算得到输出概率，按照概率对数据贴上标签；<br>\n",
    "&ensp;&ensp;&ensp;&ensp;后向计算：根据**随机梯度下降法**，更新权值，权值更新公式为：$$W_ij=W_ij+\\alpha\\delta_jx_ji$$\n",
    "&ensp;&ensp;&ensp;&ensp;其中，$\\delta_j=(t_j-y_i)y_i(1-y_i)$,$y_i$代表模型算得的标签，$t_i$代表数据的真实标签，$\\alpha$代表学习率，通常小于1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;利用$sklearn$中的库函数对原始数据进行训练，并测试训练集的精度，并根据精度要求，不断调整神经网络库函数的相关参数，直至训练集的精度达到要求为止，并利用训练好的模型去预测未知数据。<br>\n",
    "### 代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "G:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\validation.py:72: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(**kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.000875\n",
      "0.00179\n",
      "0.00273\n",
      "0.00364\n",
      "0.004555\n",
      "0.00546\n",
      "0.00638\n",
      "0.00729\n",
      "0.00824\n",
      "0.00917\n",
      "0.010095\n",
      "0.01099\n",
      "0.01194\n",
      "0.01288\n",
      "0.013845\n",
      "0.01477\n",
      "0.015675\n",
      "0.0166\n",
      "0.017515\n",
      "0.018405\n",
      "0.01933\n",
      "0.020255\n",
      "0.02118\n",
      "0.022095\n",
      "0.023035\n",
      "0.023925\n",
      "0.024855\n",
      "0.02581\n",
      "0.026725\n",
      "0.02768\n",
      "0.0286\n",
      "0.029515\n",
      "0.030445\n",
      "0.031365\n",
      "0.032305\n",
      "0.03326\n",
      "0.0342\n",
      "0.035125\n",
      "0.03606\n",
      "0.03695\n",
      "0.037885\n",
      "0.038835\n",
      "0.03977\n",
      "0.040695\n",
      "0.041605\n",
      "0.042505\n",
      "0.043445\n",
      "0.044375\n",
      "0.045315\n",
      "0.046225\n",
      "0.04718\n",
      "0.04808\n",
      "0.04903\n",
      "0.04994\n",
      "0.050865\n",
      "0.051805\n",
      "0.052735\n",
      "0.05364\n",
      "0.05455\n",
      "0.055465\n",
      "0.0564\n",
      "0.05731\n",
      "0.058235\n",
      "0.059175\n",
      "0.06013\n",
      "0.061025\n",
      "0.06193\n",
      "0.06287\n",
      "0.063805\n",
      "0.064725\n",
      "0.065635\n",
      "0.06657\n",
      "0.0675\n",
      "0.068445\n",
      "0.069365\n",
      "0.07034\n",
      "0.0713\n",
      "0.07224\n",
      "0.073165\n",
      "0.07409\n",
      "0.074995\n",
      "0.075925\n",
      "0.07685\n",
      "0.077795\n",
      "0.07871\n",
      "0.079605\n",
      "0.080545\n",
      "0.081465\n",
      "0.082365\n",
      "0.08329\n",
      "0.08422\n",
      "0.085115\n",
      "0.08603\n",
      "0.086985\n",
      "0.08792\n",
      "0.088815\n",
      "0.089735\n",
      "0.090655\n",
      "0.09157\n",
      "0.092495\n",
      "0.09342\n",
      "0.094365\n",
      "0.095285\n",
      "0.09623\n",
      "0.097155\n",
      "0.09806\n",
      "0.09897\n",
      "0.099905\n",
      "0.100815\n",
      "0.10175\n",
      "0.10265\n",
      "0.103585\n",
      "0.104515\n",
      "0.10543\n",
      "0.106345\n",
      "0.107245\n",
      "0.10814\n",
      "0.109045\n",
      "0.109955\n",
      "0.11087\n",
      "0.111785\n",
      "0.11271\n",
      "0.113605\n",
      "0.114525\n",
      "0.11545\n",
      "0.116365\n",
      "0.1173\n",
      "0.118215\n",
      "0.119125\n",
      "0.120015\n",
      "0.12095\n",
      "0.12187\n",
      "0.12282\n",
      "0.12375\n",
      "0.12466\n",
      "0.125545\n",
      "0.12649\n",
      "0.1274\n",
      "0.128365\n",
      "0.129275\n",
      "0.13019\n",
      "0.131145\n",
      "0.132085\n",
      "0.133005\n",
      "0.133925\n",
      "0.134835\n",
      "0.135795\n",
      "0.13673\n",
      "0.13764\n",
      "0.13855\n",
      "0.139435\n",
      "0.140355\n",
      "0.14128\n",
      "0.142185\n",
      "0.14313\n",
      "0.144055\n",
      "0.14494\n",
      "0.14586\n",
      "0.14675\n",
      "0.147645\n",
      "0.14861\n",
      "0.14952\n",
      "0.150445\n",
      "0.15134\n",
      "0.15227\n",
      "0.153175\n",
      "0.15411\n",
      "0.15505\n",
      "0.15599\n",
      "0.15691\n",
      "0.157845\n",
      "0.15875\n",
      "0.159655\n",
      "0.160555\n",
      "0.161485\n",
      "0.162395\n",
      "0.163325\n",
      "0.164295\n",
      "0.165235\n",
      "0.166175\n",
      "0.167125\n",
      "0.168065\n",
      "0.168985\n",
      "0.169905\n",
      "0.170805\n",
      "0.17174\n",
      "0.1727\n",
      "0.173635\n",
      "0.174565\n",
      "0.17549\n",
      "0.17642\n",
      "0.17737\n",
      "0.178315\n",
      "0.179225\n",
      "0.18015\n",
      "0.181115\n",
      "0.182065\n",
      "0.183015\n",
      "0.18393\n",
      "0.18481\n",
      "0.1857\n",
      "0.186615\n",
      "0.18753\n",
      "0.188445\n",
      "0.18937\n",
      "0.19028\n",
      "0.19121\n",
      "0.19214\n",
      "0.19309\n",
      "0.194035\n",
      "0.19497\n",
      "0.19593\n",
      "0.19686\n",
      "0.19778\n",
      "0.19871\n",
      "0.199605\n",
      "0.200545\n",
      "0.201465\n",
      "0.202375\n",
      "0.20329\n",
      "0.204235\n",
      "0.20517\n",
      "0.20608\n",
      "0.207\n",
      "0.207875\n",
      "0.208815\n",
      "0.209765\n",
      "0.210695\n",
      "0.21157\n",
      "0.212495\n",
      "0.213445\n",
      "0.214365\n",
      "0.21531\n",
      "0.21624\n",
      "0.217145\n",
      "0.21809\n",
      "0.219015\n",
      "0.21997\n",
      "0.22091\n",
      "0.221815\n",
      "0.22273\n",
      "0.22368\n",
      "0.22462\n",
      "0.22553\n",
      "0.226465\n",
      "0.22741\n",
      "0.228305\n",
      "0.22923\n",
      "0.230145\n",
      "0.231055\n",
      "0.23199\n",
      "0.23293\n",
      "0.233875\n",
      "0.234795\n",
      "0.235745\n",
      "0.236665\n",
      "0.23758\n",
      "0.238515\n",
      "0.239425\n",
      "0.24037\n",
      "0.241315\n",
      "0.242215\n",
      "0.24317\n",
      "0.24407\n",
      "0.244975\n",
      "0.245895\n",
      "0.246805\n",
      "0.24775\n",
      "0.248675\n",
      "0.24961\n",
      "0.25054\n",
      "0.25147\n",
      "0.252405\n",
      "0.25334\n",
      "0.25427\n",
      "0.255165\n",
      "0.256095\n",
      "0.25703\n",
      "0.25794\n",
      "0.258865\n",
      "0.259775\n",
      "0.26069\n",
      "0.261605\n",
      "0.262545\n",
      "0.26346\n",
      "0.26438\n",
      "0.26531\n",
      "0.266255\n",
      "0.26719\n",
      "0.268135\n",
      "0.26905\n",
      "0.269955\n",
      "0.27088\n",
      "0.2718\n",
      "0.27272\n",
      "0.27365\n",
      "0.274605\n",
      "0.275545\n",
      "0.27648\n",
      "0.27738\n",
      "0.278275\n",
      "0.279205\n",
      "0.28013\n",
      "0.281045\n",
      "0.28199\n",
      "0.28292\n",
      "0.28385\n",
      "0.28476\n",
      "0.285685\n",
      "0.286625\n",
      "0.28755\n",
      "0.288505\n",
      "0.28947\n",
      "0.29043\n",
      "0.291325\n",
      "0.292245\n",
      "0.29318\n",
      "0.29408\n",
      "0.29498\n",
      "0.29591\n",
      "0.29685\n",
      "0.29778\n",
      "0.29869\n",
      "0.299645\n",
      "0.300545\n",
      "0.30149\n",
      "0.30243\n",
      "0.30336\n",
      "0.304305\n",
      "0.305215\n",
      "0.30615\n",
      "0.307085\n",
      "0.307985\n",
      "0.308895\n",
      "0.3098\n",
      "0.310705\n",
      "0.311645\n",
      "0.312545\n",
      "0.313475\n",
      "0.31436\n",
      "0.315285\n",
      "0.316205\n",
      "0.317155\n",
      "0.318055\n",
      "0.318995\n",
      "0.319935\n",
      "0.32084\n",
      "0.321755\n",
      "0.32269\n",
      "0.32361\n",
      "0.324555\n",
      "0.325495\n",
      "0.32643\n",
      "0.327395\n",
      "0.328335\n",
      "0.329265\n",
      "0.33021\n",
      "0.331135\n",
      "0.332075\n",
      "0.33301\n",
      "0.33393\n",
      "0.334845\n",
      "0.335735\n",
      "0.336665\n",
      "0.33757\n",
      "0.338495\n",
      "0.33943\n",
      "0.340335\n",
      "0.34126\n",
      "0.342185\n",
      "0.34307\n",
      "0.34398\n",
      "0.34489\n",
      "0.345845\n",
      "0.346765\n",
      "0.3477\n",
      "0.34862\n",
      "0.349555\n",
      "0.35052\n",
      "0.35141\n",
      "0.35231\n",
      "0.353255\n",
      "0.35422\n",
      "0.35514\n",
      "0.35605\n",
      "0.35699\n",
      "0.357905\n",
      "0.358815\n",
      "0.35977\n",
      "0.360695\n",
      "0.36165\n",
      "0.36257\n",
      "0.36349\n",
      "0.364425\n",
      "0.365375\n",
      "0.366285\n",
      "0.3672\n",
      "0.368135\n",
      "0.36905\n",
      "0.369985\n",
      "0.37091\n",
      "0.371875\n",
      "0.372815\n",
      "0.37373\n",
      "0.37468\n",
      "0.37563\n",
      "0.37657\n",
      "0.377495\n",
      "0.37843\n",
      "0.379355\n",
      "0.380285\n",
      "0.38122\n",
      "0.382185\n",
      "0.38312\n",
      "0.384015\n",
      "0.38494\n",
      "0.38584\n",
      "0.38679\n",
      "0.38769\n",
      "0.38859\n",
      "0.389535\n",
      "0.390485\n",
      "0.391385\n",
      "0.392325\n",
      "0.393265\n",
      "0.39417\n",
      "0.395115\n",
      "0.396025\n",
      "0.396945\n",
      "0.39788\n",
      "0.3988\n",
      "0.399735\n",
      "0.400645\n",
      "0.401525\n",
      "0.40244\n",
      "0.403355\n",
      "0.404265\n",
      "0.40516\n",
      "0.40609\n",
      "0.407015\n",
      "0.407945\n",
      "0.40889\n",
      "0.409825\n",
      "0.41073\n",
      "0.41168\n",
      "0.41262\n",
      "0.413535\n",
      "0.41447\n",
      "0.41539\n",
      "0.41629\n",
      "0.41719\n",
      "0.418085\n",
      "0.419015\n",
      "0.419935\n",
      "0.42089\n",
      "0.42183\n",
      "0.42278\n",
      "0.423695\n",
      "0.424585\n",
      "0.42552\n",
      "0.426445\n",
      "0.427355\n",
      "0.428275\n",
      "0.42923\n",
      "0.430145\n",
      "0.431035\n",
      "0.431945\n",
      "0.432865\n",
      "0.433815\n",
      "0.43476\n",
      "0.435685\n",
      "0.43659\n",
      "0.437545\n",
      "0.438455\n",
      "0.43938\n",
      "0.44034\n",
      "0.44124\n",
      "0.442175\n",
      "0.443105\n",
      "0.44405\n",
      "0.444995\n",
      "0.445925\n",
      "0.44684\n",
      "0.44776\n",
      "0.44869\n",
      "0.449615\n",
      "0.450515\n",
      "0.45144\n",
      "0.452335\n",
      "0.453275\n",
      "0.45419\n",
      "0.45513\n",
      "0.45603\n",
      "0.456935\n",
      "0.457885\n",
      "0.45883\n",
      "0.459735\n",
      "0.46066\n",
      "0.461595\n",
      "0.462505\n",
      "0.463425\n",
      "0.464375\n",
      "0.465335\n",
      "0.466265\n",
      "0.46722\n",
      "0.46815\n",
      "0.46905\n",
      "0.46995\n",
      "0.47085\n",
      "0.47176\n",
      "0.47267\n",
      "0.47358\n",
      "0.474495\n",
      "0.47545\n",
      "0.476345\n",
      "0.47728\n",
      "0.4782\n",
      "0.47914\n",
      "0.48009\n",
      "0.48103\n",
      "0.481955\n",
      "0.482855\n",
      "0.4838\n",
      "0.484705\n",
      "0.485635\n",
      "0.486555\n",
      "0.48747\n",
      "0.48838\n",
      "0.489305\n",
      "0.490205\n",
      "0.4911\n",
      "0.49204\n",
      "0.49294\n",
      "0.493875\n",
      "0.494795\n",
      "0.49573\n",
      "0.496615\n",
      "0.497505\n",
      "0.498445\n",
      "0.499345\n",
      "0.5003\n",
      "0.501215\n",
      "0.50215\n",
      "0.503085\n",
      "0.504005\n",
      "0.50493\n",
      "0.50584\n",
      "0.50678\n",
      "0.50769\n",
      "0.508615\n",
      "0.50956\n",
      "0.510455\n",
      "0.511395\n",
      "0.51232\n",
      "0.513235\n",
      "0.51417\n",
      "0.51508\n",
      "0.515975\n",
      "0.516905\n",
      "0.51782\n",
      "0.51874\n",
      "0.519665\n",
      "0.52059\n",
      "0.521505\n",
      "0.522405\n",
      "0.52335\n",
      "0.52427\n",
      "0.525225\n",
      "0.526115\n",
      "0.527055\n",
      "0.527995\n",
      "0.52891\n",
      "0.52983\n",
      "0.53077\n",
      "0.53165\n",
      "0.53258\n",
      "0.5335\n",
      "0.534435\n",
      "0.535345\n",
      "0.53628\n",
      "0.53721\n",
      "0.538145\n",
      "0.53905\n",
      "0.53997\n",
      "0.54091\n",
      "0.541815\n",
      "0.54273\n",
      "0.543675\n",
      "0.544645\n",
      "0.54555\n",
      "0.546495\n",
      "0.54742\n",
      "0.54833\n",
      "0.54927\n",
      "0.55018\n",
      "0.551105\n",
      "0.552035\n",
      "0.55296\n",
      "0.553905\n",
      "0.55482\n",
      "0.555765\n",
      "0.55668\n",
      "0.557595\n",
      "0.55852\n",
      "0.55947\n",
      "0.5604\n",
      "0.561345\n",
      "0.56228\n",
      "0.56317\n",
      "0.564085\n",
      "0.565015\n",
      "0.565935\n",
      "0.566855\n",
      "0.567775\n",
      "0.5687\n",
      "0.56963\n",
      "0.570555\n",
      "0.57148\n",
      "0.572405\n",
      "0.5733\n",
      "0.57423\n",
      "0.575185\n",
      "0.576105\n",
      "0.577035\n",
      "0.57795\n",
      "0.5789\n",
      "0.579785\n",
      "0.580665\n",
      "0.5816\n",
      "0.58257\n",
      "0.583485\n",
      "0.584405\n",
      "0.58536\n",
      "0.58628\n",
      "0.58721\n",
      "0.58815\n",
      "0.58909\n",
      "0.590055\n",
      "0.59097\n",
      "0.5919\n",
      "0.592825\n",
      "0.59375\n",
      "0.59467\n",
      "0.5956\n",
      "0.59652\n",
      "0.597475\n",
      "0.5984\n",
      "0.599345\n",
      "0.60027\n",
      "0.6012\n",
      "0.602095\n",
      "0.60299\n",
      "0.603925\n",
      "0.60487\n",
      "0.60581\n",
      "0.60672\n",
      "0.607695\n",
      "0.608625\n",
      "0.60954\n",
      "0.61047\n",
      "0.61141\n",
      "0.612355\n",
      "0.61331\n",
      "0.614245\n",
      "0.615185\n",
      "0.6161\n",
      "0.617045\n",
      "0.61795\n",
      "0.61889\n",
      "0.61982\n",
      "0.620735\n",
      "0.62166\n",
      "0.622575\n",
      "0.623505\n",
      "0.62443\n",
      "0.62536\n",
      "0.62627\n",
      "0.62718\n",
      "0.628115\n",
      "0.629025\n",
      "0.62993\n",
      "0.630855\n",
      "0.63176\n",
      "0.63269\n",
      "0.63362\n",
      "0.634565\n",
      "0.635495\n",
      "0.636425\n",
      "0.63736\n",
      "0.63831\n",
      "0.63921\n",
      "0.640115\n",
      "0.64103\n",
      "0.641965\n",
      "0.642885\n",
      "0.64383\n",
      "0.644765\n",
      "0.6457\n",
      "0.646635\n",
      "0.64754\n",
      "0.648475\n",
      "0.649395\n",
      "0.65036\n",
      "0.65131\n",
      "0.652275\n",
      "0.65322\n",
      "0.654135\n",
      "0.65507\n",
      "0.655975\n",
      "0.65692\n",
      "0.65785\n",
      "0.658785\n",
      "0.659715\n",
      "0.660625\n",
      "0.661525\n",
      "0.66244\n",
      "0.66336\n",
      "0.664255\n",
      "0.66519\n",
      "0.66611\n",
      "0.667025\n",
      "0.66795\n",
      "0.6689\n",
      "0.66979\n",
      "0.670695\n",
      "0.671595\n",
      "0.672545\n",
      "0.673485\n",
      "0.674415\n",
      "0.67533\n",
      "0.67625\n",
      "0.677195\n",
      "0.678095\n",
      "0.67902\n",
      "0.67996\n",
      "0.680895\n",
      "0.681815\n",
      "0.68275\n",
      "0.68368\n",
      "0.6846\n",
      "0.68552\n",
      "0.686465\n",
      "0.687405\n",
      "0.688315\n",
      "0.68926\n",
      "0.690175\n",
      "0.691095\n",
      "0.692025\n",
      "0.692955\n",
      "0.693845\n",
      "0.694745\n",
      "0.69568\n",
      "0.69662\n",
      "0.697515\n",
      "0.69846\n",
      "0.69939\n",
      "0.700345\n",
      "0.70128\n",
      "0.702195\n",
      "0.703095\n",
      "0.70401\n",
      "0.70497\n",
      "0.705875\n",
      "0.706755\n",
      "0.70768\n",
      "0.70861\n",
      "0.709545\n",
      "0.710485\n",
      "0.711375\n",
      "0.712295\n",
      "0.713245\n",
      "0.71414\n",
      "0.715065\n",
      "0.715985\n",
      "0.716925\n",
      "0.71784\n",
      "0.718775\n",
      "0.7197\n",
      "0.72063\n",
      "0.72157\n",
      "0.722515\n",
      "0.72345\n",
      "0.724365\n",
      "0.72532\n",
      "0.72623\n",
      "0.727135\n",
      "0.72804\n",
      "0.72897\n",
      "0.729905\n",
      "0.730825\n",
      "0.731755\n",
      "0.73266\n",
      "0.73354\n",
      "0.73449\n",
      "0.73542\n",
      "0.736365\n",
      "0.73726\n",
      "0.738175\n",
      "0.73911\n",
      "0.740055\n",
      "0.740975\n",
      "0.741905\n",
      "0.742845\n",
      "0.743755\n",
      "0.7447\n",
      "0.74564\n",
      "0.74655\n",
      "0.74747\n",
      "0.7484\n",
      "0.74933\n",
      "0.75024\n",
      "0.751175\n",
      "0.7521\n",
      "0.752985\n",
      "0.75391\n",
      "0.754825\n",
      "0.75575\n",
      "0.756705\n",
      "0.757635\n",
      "0.75861\n",
      "0.75954\n",
      "0.760465\n",
      "0.76137\n",
      "0.7623\n",
      "0.763245\n",
      "0.76415\n",
      "0.76507\n",
      "0.76599\n",
      "0.76693\n",
      "0.767855\n",
      "0.768795\n",
      "0.76973\n",
      "0.770645\n",
      "0.771555\n",
      "0.77249\n",
      "0.773445\n",
      "0.774355\n",
      "0.775295\n",
      "0.776225\n",
      "0.777175\n",
      "0.77809\n",
      "0.77902\n",
      "0.77993\n",
      "0.78087\n",
      "0.781825\n",
      "0.782785\n",
      "0.78369\n",
      "0.78459\n",
      "0.78553\n",
      "0.78647\n",
      "0.78738\n",
      "0.788305\n",
      "0.789235\n",
      "0.79015\n",
      "0.79109\n",
      "0.79201\n",
      "0.792965\n",
      "0.79388\n",
      "0.79479\n",
      "0.79572\n",
      "0.796635\n",
      "0.797545\n",
      "0.7985\n",
      "0.79941\n",
      "0.800335\n",
      "0.801275\n",
      "0.8022\n",
      "0.80311\n",
      "0.804055\n",
      "0.80498\n",
      "0.805905\n",
      "0.806835\n",
      "0.80776\n",
      "0.808675\n",
      "0.809625\n",
      "0.810555\n",
      "0.811495\n",
      "0.81242\n",
      "0.81338\n",
      "0.81432\n",
      "0.815245\n",
      "0.816175\n",
      "0.81712\n",
      "0.818045\n",
      "0.81898\n",
      "0.819915\n",
      "0.82083\n",
      "0.821755\n",
      "0.82269\n",
      "0.823625\n",
      "0.824565\n",
      "0.82551\n",
      "0.826445\n",
      "0.82733\n",
      "0.828245\n",
      "0.829195\n",
      "0.8301\n",
      "0.831045\n",
      "0.831925\n",
      "0.83287\n",
      "0.833785\n",
      "0.8347\n",
      "0.83562\n",
      "0.83654\n",
      "0.837475\n",
      "0.838405\n",
      "0.839305\n",
      "0.840205\n",
      "0.841105\n",
      "0.842045\n",
      "0.842995\n",
      "0.84393\n",
      "0.84486\n",
      "0.845755\n",
      "0.846675\n",
      "0.84761\n",
      "0.84854\n",
      "0.849505\n",
      "0.850445\n",
      "0.851335\n",
      "0.852305\n",
      "0.853215\n",
      "0.854145\n",
      "0.85508\n",
      "0.85604\n",
      "0.85695\n",
      "0.857885\n",
      "0.858805\n",
      "0.85973\n",
      "0.86065\n",
      "0.86158\n",
      "0.862475\n",
      "0.8634\n",
      "0.864315\n",
      "0.865235\n",
      "0.86616\n",
      "0.86709\n",
      "0.868\n",
      "0.86892\n",
      "0.869865\n",
      "0.87078\n",
      "0.871695\n",
      "0.87263\n",
      "0.87356\n",
      "0.87445\n",
      "0.87539\n",
      "0.876285\n",
      "0.877225\n",
      "0.878155\n",
      "0.87905\n",
      "0.879975\n",
      "0.880875\n",
      "0.881815\n",
      "0.88274\n",
      "0.88366\n",
      "0.88459\n",
      "0.885515\n",
      "0.886435\n",
      "0.887365\n",
      "0.88831\n",
      "0.88925\n",
      "0.890185\n",
      "0.891095\n",
      "0.89199\n",
      "0.89292\n",
      "0.893805\n",
      "0.894705\n",
      "0.89562\n",
      "0.89653\n",
      "0.89744\n",
      "0.898405\n",
      "0.89935\n",
      "0.90029\n",
      "0.90121\n",
      "0.902115\n",
      "0.903045\n",
      "0.90398\n",
      "0.90489\n",
      "0.905805\n",
      "0.906725\n",
      "0.90765\n",
      "0.908565\n",
      "0.90947\n",
      "0.91041\n",
      "0.91133\n",
      "0.912265\n",
      "0.913155\n",
      "0.91406\n",
      "0.915\n",
      "0.915905\n",
      "0.916875\n",
      "0.917835\n",
      "0.918765\n",
      "0.919695\n",
      "0.920605\n",
      "0.92155\n",
      "0.92247\n",
      "0.92339\n",
      "0.924315\n",
      "0.925235\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "num,q1,q2,q3,q4,q5,q6,q7,q8,q9,q10,q11,q12,q13,q14,q15,q16,q17,q18,q19,q20,q21,q22,q23,q24,q25,q26,q27,q28,q29,q30,q31,q32,q33,q34,q35,q36,y= np.loadtxt(\"train.csv\",delimiter=',', usecols=(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37), \n",
    "\tunpack=True)\n",
    "num = num.reshape(-1,1)\n",
    "y = y.reshape(-1,1)\n",
    "for i in range(1,37):\n",
    "    exec (\"q%s=q%s.reshape(-1,1)\"%(i,i))\n",
    "train = np.hstack((q1.reshape(-1,1),q2.reshape(-1,1)))\n",
    "for i in range(3,37):\n",
    "    exec (\"train=np.hstack((train,q%s.reshape(-1,1)))\"%i)\n",
    "#产生训练集\n",
    "\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "clf_class= MLPClassifier(solver='adam', alpha=1e-5,hidden_layer_sizes=(40,20), random_state=1,max_iter=1500)\n",
    "clf_class.fit(train,y)\n",
    "#sklearn神经网络学习，隐藏层两层，第一层40个节点，第二层20个节点，最多迭代1500次\n",
    "\n",
    "y_pred=[]\n",
    "j=0\n",
    "for i in range(np.shape(train)[0]):\n",
    "    y_pred.append(clf_class.predict([train[i]]))\n",
    "for i in range(np.shape(train)[0]):\n",
    "    if y_pred[i]==y[i]:\n",
    "        j=j+1\n",
    "    if (i+1)%200==0:\n",
    "        print(j/200000)\n",
    "#训练集准确率计算：0.925\n",
    "\n",
    "Q1,Q2,Q3,Q4,Q5,Q6,Q7,Q8,Q9,Q10,Q11,Q12,Q13,Q14,Q15,Q16,Q17,Q18,Q19,Q20,Q21,Q22,Q23,Q24,Q25,Q26,Q27,Q28,Q29,Q30,Q31,Q32,Q33,Q34,Q35,Q36= np.loadtxt(\"test.csv\",delimiter=',', usecols=(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36), \n",
    "\tunpack=True)\n",
    "for i in range(1,37):\n",
    "    exec (\"Q%s=Q%s.reshape(-1,1)\"%(i,i))\n",
    "test = np.hstack((Q1.reshape(-1,1),Q2.reshape(-1,1)))\n",
    "for i in range(3,37):\n",
    "    exec (\"test=np.hstack((test,Q%s.reshape(-1,1)))\"%i)\n",
    "#产生测试集\n",
    "\n",
    "y_pred=[]\n",
    "j=0\n",
    "for i in range(np.shape(test)[0]):\n",
    "    y_pred.append(clf_class.predict([test[i]]))\n",
    "#对测试集贴标签\n",
    "\n",
    "import csv\n",
    "f = open('sample_submit.csv','w',encoding='utf-8',newline='' \"\")\n",
    "csv_writer = csv.writer(f)\n",
    "csv_writer.writerow([\"Caseld\",\"Evaluation\"])\n",
    "for i in range(80000):\n",
    "    n=200000+i+1\n",
    "    csv_writer.writerow([n,y_pred[i]])\n",
    "f.close()\n",
    "#数据写入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Caseld Evaluation\n",
      "0   200001       [0.]\n",
      "1   200002       [0.]\n",
      "2   200003       [0.]\n",
      "3   200004       [1.]\n",
      "4   200005       [0.]\n",
      "..     ...        ...\n",
      "95  200096       [0.]\n",
      "96  200097       [1.]\n",
      "97  200098       [0.]\n",
      "98  200099       [0.]\n",
      "99  200100       [0.]\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "sheet = pd.read_csv('sample_submit_rd.csv',encoding='utf-8')\n",
    "print(sheet.head(100))\n",
    "#打印表格前100行作为举例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 错误数据的可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;错误数据的可视化采用混淆矩阵的方式呈现，主对角线元素代表分类正确的元素个数；副对角线元素代表分类错误的数据，并将得到的混淆矩阵用热力图进行可视化处理，方便观察。<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAFzCAYAAAAwtX3sAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuUElEQVR4nO3deZgdZZn38e+dhCUC2YCEmKBEiCIwKiAh6qsvipK4QFgChDVCxiCCgCAKMiMjvCgoiiKCE2QHCZE14LBkQEQUAggim0gkDoRsQEIIe5b7/eNUmFNNp9PpzulOur4fr7r6nKfqqXrOuez0j7ueqorMRJIkqeq6dfYAJEmSVgeGIkmSJAxFkiRJgKFIkiQJMBRJkiQBhiJJkiQAenT2AJan57ZHeq8AqRPMv/+czh6CVFnr9iA68njt/Vv7+kPndOh4G221DUWSJKnBwhNG9fw2JEmSsFIkSVJ1RZc6+9VuVookSaqq6Na+ZUW7j7gwIuZGxKNN2r8eEU9GxGMR8cO69hMjYlqxbkRd+/YR8Uix7uyIWpqLiHUi4qqifWpEbFbXZ2xEPFUsY1vzdRiKJEmqqoj2LSt2MTCyfMj4NDAK+FBmbg2cWbRvBYwBti76nBsR3Ytu5wHjgaHFsmyf44D5mbkFcBZwRrGvfsDJwI7AMODkiOi7osEaiiRJUkNk5l3AvCbNhwOnZ+abxTZzi/ZRwMTMfDMzpwPTgGERMRDolZn3ZO0p9pcCu9f1uaR4fTWwc1FFGgFMycx5mTkfmEKTcNYcQ5EkSVXVztNnETE+Ih6oW8a34qjvBz5ZnO76fUTsULQPAp6t225G0TaoeN20vdQnMxcDC4ANW9hXi5xoLUlSVbVzonVmTgAmrGS3HkBfYDiwAzApIt4Hzd6jKVtop419lstKkSRJVdXgidbLMQO4NmvuA5YCGxXtm9ZtNxiYWbQPbqad+j4R0QPoTe103fL21SJDkSRJVdX4idbNuR74TO3w8X5gbeAFYDIwpriibAi1CdX3ZeYsYGFEDC/mCx0M3FDsazKw7Mqy0cAdxbyjW4FdIqJvMcF6l6KtRZ4+kyRJDRERVwI7ARtFxAxqV4RdCFxYXKb/FjC2CDKPRcQk4HFgMXBEZi4pdnU4tSvZegI3FwvABcBlETGNWoVoDEBmzouIU4H7i+1OycymE77fOd7aOFY/PvtM6hw++0zqPB3+7LOPf6d9zz770/e71N0frRRJklRV3tG6xFAkSVJV+UDYEkORJElVZaWoxIgoSZKElSJJkqrL02clhiJJkqrKUFRiKJIkqaq6OaeonhFRkiQJK0WSJFWXp89KDEWSJFWVl+SXGIokSaoqK0UlhiJJkqrKSlGJEVGSJAkrRZIkVZenz0oMRZIkVZWnz0oMRZIkVZWVohJDkSRJVWWlqMSIKEmShJUiSZKqy9NnJYYiSZKqytNnJYYiSZKqykpRid+GJEkSVookSaouK0UlhiJJkqrKOUUlhiJJkqrKSlGJoUiSpKqyUlRiRJQkScJKkSRJ1eXpsxJDkSRJVeXpsxJDkSRJFRWGohJDkSRJFWUoKvNkoiRJEoYiSZKqK9q5rGj3ERdGxNyIeLSZdd+MiIyIjeraToyIaRHxZESMqGvfPiIeKdadHUWJKyLWiYirivapEbFZXZ+xEfFUsYxtzddhKJIkqaIiol1LK1wMjGzmuJsCnwOeqWvbChgDbF30OTciuherzwPGA0OLZdk+xwHzM3ML4CzgjGJf/YCTgR2BYcDJEdF3RYM1FEmSVFGNDkWZeRcwr5lVZwHfArKubRQwMTPfzMzpwDRgWEQMBHpl5j2ZmcClwO51fS4pXl8N7FxUkUYAUzJzXmbOB6bQTDhrylAkSZLaJCLGR8QDdcv4VvTZDXguMx9usmoQ8Gzd+xlF26DiddP2Up/MXAwsADZsYV8t8uozSZIqqr1Xn2XmBGDCShzvXcBJwC7NrW7uEC20t7XPclkpkiSpojpgTlFTmwNDgIcj4p/AYODBiNiEWjVn07ptBwMzi/bBzbRT3yciegC9qZ2uW96+WmQokiSpqhp89VlTmflIZvbPzM0yczNq4WW7zJwNTAbGFFeUDaE2ofq+zJwFLIyI4cV8oYOBG4pdTgaWXVk2GrijmHd0K7BLRPQtJljvUrS1yNNnkiRVVKNv3hgRVwI7ARtFxAzg5My8oLltM/OxiJgEPA4sBo7IzCXF6sOpXcnWE7i5WAAuAC6LiGnUKkRjin3Ni4hTgfuL7U7JzOYmfJfHWwtUq5+e2x65eg5M6uLm339OZw9Bqqx1e7Sl/tJ2fQ64vF1/a1+64sAudUtsK0WSJFWUj/koMxRJklRRhqIyQ5EkSRVlKCozFEmSVFVmohIvyZckScJKkSRJleXpszJDkSRJFWUoKjMUSZJUUYaiMucUSZIkYaVIkqTqslBUYiiSJKmiPH1WZiiSJKmiDEVlhiJJkirKUFTmRGtJkiSsFEmSVFlWisoMRZIkVZWZqMRQJElSRVkpKjMUSZJUUYaiMidaS5IkYaVIkqTKslJUZiiSJKmqzEQlhiJJkirKSlGZc4okSZKwUiTglycfwOc/tQ3Pz1vIR/f+frPbfHL7ofzo+L1Yq0d3XnzpFXb515+165hrr9WDC049iG0/+B7mLXiVA799Ic/MmgfAKw+czaPTZgLw7Oz57H3Mf7brWNLqbsmSJey3z170HzCAc84t//994cKFfOfbxzN71kwWL1nC2EMOZfc99mrX8d566y1OOvFbPPHYY/Tu04cf/vgsBg0azMyZz3Hs0V9n6ZIlLFq8mP0OOJB99t2vXcfS6s1KUZmVInHZjfcy6ohfLHd97/V78rPv7MPex/wn248+jQOOv6DV+37PwH7cev7R72j/8u4fY/7C19lm1Pf4+RW/47SjR7297vU3FzF8zOkMH3O6gUiVcMVll/K+923e7LqrrryC922+Ob+5bjIXXHwZP/7hGSx6661W7fe552Yw7ssHvaP9umt+Q69evbjplikcePCX+elPzgRg44025tIrJjLp2hu44spJXPSr85k7d07bP5hWexHRrqWraVgoiogtI+LbEXF2RPyseP3BRh1PbffHB//BvAWvLXf9vp//KDfc/jDPzp4PwPPzX3l73Zgv7MAfLvsm9048gZ+fNIZu3Vr3S/KlnT7EFTdOBeDa/36InYZ9oB2fQFpzzZk9mz/cdSd77DW62fURwWuvvkpm8tprr9K7d2+696gV+W+68Qb233c0++w5ilP+47ssWbKkVcf83R13sNuoPQD43C4juO/ee8hM1lp7bdZee20A3lr0FkuXLl0Fn1CrM0NRWUNCUUR8G5hIbV77fcD9xesrI+KERhxTjTP0vf3p0+td3Hr+0fzxim+x/5eGAfCBIQMYvct2fPqQnzB8zOksWbqUMV/YoVX7fHf/3swoQtaSJUt5+ZXX2bDPegCsu3YP7r7iW/z+kuPYdacPNeZDSauJH57+fb5x3PF069b8P8dj9j+Ap5/+B5/d6ZOM3n03vnXiSXTr1o2n//EPbr35Zi65/EomXXsD3bt1479uurFVx5w7dw6bbDIQgB49erD+Bhvw0ku138fZs2Yxeo9dGbHzThwy7iv07z9g1XxQrZ6inUsX06g5ReOArTNzUX1jRPwEeAw4vblOETEeGA/QY/BO9Nho6wYNTyujR/dubPfBTfn8YT+n57prceclx3HfX//Jp4d9gO22eg93X/4tAHqusxbPz6tVka768Vd476ANWXut7my6ST/unVjLwr/49Z1cNvneZv8LI7P28/1f+C6znl/AZoM25JYJR/HotJlMn/FCx3xYqQP9/s7f0a9fP7baehvuv29qs9v86e672XLLD/Kriy7l2Wee4bCvHMJ223+UqffewxOPP8oB+9YqTG+8+Qb9NtwQgGOOOoKZM2awaNEiZs2axT571k5P73/Qwey+x17ksl+2Ost+JzcZOJCrr7uRuXPncMzXj+Bzu4xgw402asTHl1Y7jQpFS4F3A//TpH1gsa5ZmTkBmADQc9sj3/lbq07x3NyXeOGlV3ntjbd47Y23uPvBaXzo/YOICC6/cSrf/fnkd/TZ97jzgdqcovNPOYgRXylPzH5uzksM3qQvz819ie7du9Fr/Z7MW/AqALOeXwDAP597kbseeIqPbDnYUKQu6S8PPcidd97B3X+4izfffJNXX32FE7/9TX5wxplvb3PD9ddy6L+OJyJ4z3vfy6BBg5n+9NMkya6j9uDobxz3jv3+9OzaHMHnnpvBd086kQsuvqy0fsCATZg9exYDNtmExYsX88rChfTu3ae0Tf/+A9h8i6E8+OcH+NyIkav+w2u10BVPgbVHo+YUHQPcHhE3R8SEYrkFuB1456xbrdZuvPOvfGLbzenevRs9112LHbbZjL9Nn83v7nuSPT77ETbuuz4AfXu9i/cM7Nuqff72949wwK47ArDnZ7fl9/f/HYA+G/Rk7bVqWX3DPuvxsY+8jyeent2ATyV1vqO/cRxT7riLm6fcwRln/oQddhxeCkRQq9xMvfceAF584QX++c/pDN50MDvu+DH++7ZbefHFFwFY8NJLzJz5XKuOu9OnP8PkG64DYMpttzJsx+FEBHNmz+aNN94A4OUFC/jLQw+y2ZAhq+rjajXknKKyhlSKMvOWiHg/MAwYRO3M4wzg/sxs3UxAdZhLfvBlPrn9UDbqsz7TbjmVU3/5X6zVozsAv7r6bp6cPocpf3qc+yedyNKlycXX/YnH/zELgO/94iZuPO9IukWwaPESvnH6JJ6ZNX+Fx7z4+j9x4f87mEdvOJn5L7/KQSdcBMCW79uEn5+0H0tzKd2iG2deNIW/GYpUMZOuuhKAffbdj/Ff/Rr/ftKJ7LX7rmQmxxz7Tfr27Uffvv044qhjOPwrh7I0l9Kjx1p859++y7vfPWiF+99jr9GcdMLxfGnk5+jVuzc/PPMsAJ5++h/8+EenEwRJMvbLhzL0/V4E0ZV1wVzTLtHcueXVgafPpM4x//5zOnsIUmWt26Njpy9v8c2b2/W3dtqZn+9Sscr7FEmSVFGNPn0WERdGxNyIeLSu7UcR8beI+GtEXBcRferWnRgR0yLiyYgYUde+fUQ8Uqw7O4qDR8Q6EXFV0T41Ijar6zM2Ip4qlrGt+T4MRZIkVVRE+5ZWuBhoOlN/CrBNZn4I+DtwYm0ssRUwBti66HNuRHQv+pxH7er0ocWybJ/jgPmZuQVwFnBGsa9+wMnAjtSm8pwcESuc9GookiSpohpdKcrMu4B5Tdpuy8zFxdt7gcHF61HAxMx8MzOnA9OAYRExEOiVmfdkbc7PpcDudX0uKV5fDexcVJFGAFMyc15mzqcWxFZ4GaWhSJKkimpvpSgixkfEA3XL+JUcwqHAzcXrQcCzdetmFG2DitdN20t9iqC1ANiwhX21yAfCSpKkNqm/v+DKioiTgMXAFcuamjtEC+1t7bNcVookSaqobt2iXUtbFROfvwQckP97GfwMYNO6zQYDM4v2wc20l/pERA+gN7XTdcvbV4sMRZIkVVQHTLRu5pgxEvg2sFtm1j+NfDIwpriibAi1CdX3ZeYsYGFEDC/mCx0M3FDXZ9mVZaOBO4qQdSuwS0T0LSZY71K0tcjTZ5IkVVSj70odEVcCOwEbRcQMaleEnQisA0wpjn9vZn41Mx+LiEnA49ROqx1Rd8Pnw6ldydaT2hykZfOQLgAui4hp1CpEYwAyc15EnErtgfQAp2RmacJ3cwxFkiRVVKPvaJ2Z+zXTfEEL258GnNZM+wPANs20vwHsvZx9XQhc2OrB4ukzSZIkwEqRJEmV1RUf6toehiJJkirKUFRmKJIkqaLMRGXOKZIkScJKkSRJleXpszJDkSRJFWUmKjMUSZJUUVaKygxFkiRVlJmozInWkiRJWCmSJKmyPH1WZiiSJKmizERlhiJJkirKSlGZoUiSpIoyE5U50VqSJAkrRZIkVZanz8oMRZIkVZSZqMxQJElSRVkpKnNOkSRJElaKJEmqLAtFZYYiSZIqytNnZYYiSZIqylBUZiiSJKmizERlTrSWJEnCSpEkSZXl6bMyQ5EkSRVlJiozFEmSVFFWisoMRZIkVZSZqMyJ1pIkSVgpkiSpsrpZKioxFEmSVFFmojJDkSRJFeVE6zLnFEmSpIaIiAsjYm5EPFrX1i8ipkTEU8XPvnXrToyIaRHxZESMqGvfPiIeKdadHUWai4h1IuKqon1qRGxW12dscYynImJsa8ZrKJIkqaK6RfuWVrgYGNmk7QTg9swcCtxevCcitgLGAFsXfc6NiO5Fn/OA8cDQYlm2z3HA/MzcAjgLOKPYVz/gZGBHYBhwcn34Wu730aqPJEmSupyIaNeyIpl5FzCvSfMo4JLi9SXA7nXtEzPzzcycDkwDhkXEQKBXZt6TmQlc2qTPsn1dDexcVJFGAFMyc15mzgem8M5w9g6GIkmSKiqivUuMj4gH6pbxrTjsgMycBVD87F+0DwKerdtuRtE2qHjdtL3UJzMXAwuADVvYV4ucaC1JUkUF7ZtonZkTgAmrZjTNDiZbaG9rn+WyUiRJkjrSnOKUGMXPuUX7DGDTuu0GAzOL9sHNtJf6REQPoDe103XL21eLDEWSJFVUB0y0bs5kYNnVYGOBG+raxxRXlA2hNqH6vuIU28KIGF7MFzq4SZ9l+xoN3FHMO7oV2CUi+hYTrHcp2lrk6TNJkiqq0fcpiogrgZ2AjSJiBrUrwk4HJkXEOOAZYG+AzHwsIiYBjwOLgSMyc0mxq8OpXcnWE7i5WAAuAC6LiGnUKkRjin3Ni4hTgfuL7U7JzKYTvt/BUCRJUkU1+t6NmbnfclbtvJztTwNOa6b9AWCbZtrfoAhVzay7ELiw1YPFUCRJUmX57LOylZpTVJyb+1CjBiNJktRZVhiKIuLOiOhV3B3yYeCiiPhJ44cmSZIaqb33KepqWlMp6p2ZLwN7Ahdl5vbAZxs7LEmS1GiNvqP1mqY1oahHcR+BfYCbGjweSZLUQawUlbUmFJ1C7dr+aZl5f0S8D3iqscOSJEnqWCu8+iwzfwP8pu7908BejRyUJElqPK8+K1tuKIqIn9PCc0Iy86iGjEiSJHUII1FZS5WiBzpsFJIkqcN1xcnS7bHcUJSZl9S/j4j1MvPVxg9JkiR1hHY8v6xLas19ij4WEY8DTxTvPxwR5zZ8ZJIkSR2oNVef/RQYAbwIkJkPA59q4JgkSVIH8D5FZa169llmPtvkwy9Z3raSJGnN0AVzTbu0JhQ9GxEfBzIi1gaOojiVJkmS1lxdsdrTHq0JRV8FfgYMAp6jdiPHIxo5KEmS1HhOtC5rzc0bXwAO6ICxSJIkdZrWXH32voi4MSKej4i5EXFD8agPSZK0BnOidVlrrj77NTAJGAi8m9ojP65s5KAkSVLjRTuXrqY1oSgy87LMXFwsl9PC4z8kSdKaoVtEu5aupqVnn/UrXv4uIk4AJlILQ/sCv+2AsUmSJHWYliZa/5laCFoWBQ+rW5fAqY0alCRJarwuWOxpl5aefTakIwciSZI6VlecLN0erbqjdURsA2wFrLusLTMvbdSgJElS45mJylYYiiLiZGAnaqHov4DPA3cDhiJJktZgXXGydHu05uqz0cDOwOzMPAT4MLBOQ0clSZLUwVpz+uz1zFwaEYsjohcwF/DmjZIkreEsFJW1JhQ9EBF9gPOpXZH2CnBfIwcFMOeesxt9CEnNeGHhm509BKmyBvft2BMxTrQua82zz75WvPxlRNwC9MrMvzZ2WJIkqdFaM4emSlq6eeN2La3LzAcbMyRJktQRrBSVtVQp+nEL6xL4zCoeiyRJUqdp6eaNn+7IgUiSpI7VzUJRSatu3ihJkroeQ1GZc6wkSaqoiGjX0or9fyMiHouIRyPiyohYNyL6RcSUiHiq+Nm3bvsTI2JaRDwZESPq2rePiEeKdWdHcfCIWCcirirap0bEZu35PgxFkiRplYuIQcBRwEczcxugOzAGOAG4PTOHArcX74mIrYr1WwMjgXMjonuxu/OA8cDQYhlZtI8D5mfmFsBZwBntGfMKQ1HUHBgR3y3evycihrXnoJIkqfN1i/YtrdAD6BkRPYB3ATOBUcAlxfpLgN2L16OAiZn5ZmZOB6YBwyJiILXbAd2TmUntMWP1fZbt62pg52VVpDZ9H63Y5lzgY8B+xfuFwC/aekBJkrR6iGjvEuMj4oG6ZfyyfWfmc8CZwDPALGBBZt4GDMjMWcU2s4D+RZdBwLN1w5tRtA0qXjdtL/XJzMXAAmDDtn4frZlovWNmbhcRDxUHnR8Ra7f1gJIkafXQ3gfCZuYEYEJz64q5QqOAIcBLwG8i4sAWdtfcYLKF9pb6tElrKkWLinN6CRARGwNL23pASZK0eujWzmUFPgtMz8znM3MRcC3wcWBOcUqM4ufcYvsZwKZ1/QdTO902o3jdtL3UpzhF1xuY17pP/06tCUVnA9cB/SPiNOBu4PttPaAkSaqEZ4DhEfGuYp7PzsATwGRgbLHNWOCG4vVkYExxRdkQahOq7ytOsS2MiOHFfg5u0mfZvkYDdxTzjtqkNc8+uyIi/lx8mAB2z8wn2npASZK0emjkUz4yc2pEXA08CCwGHqJ2qm19YFJEjKMWnPYutn8sIiYBjxfbH5GZS4rdHQ5cDPQEbi4WgAuAyyJiGrUK0Zj2jDlWFKgi4j3NtWfmM+058Iq8/MbSNic9SW338uuLOnsIUmUN7rtOh95O8d9veapdf2tPHTm0S93+sTUTrX/L/050WpfahKknqd1HQJIkraF8HmxZa06f/Uv9+4jYDjisYSOSJEnqBCv97LPMfDAidmjEYCRJUsfx2WdlKwxFEXFs3dtuwHbA8w0bkSRJ6hDtvU9RV9OaStEGda8XU5tjdE1jhiNJkjqKmaisxVBU3LRx/cw8voPGI0mSOoinz8qWe/PGiOhR3B9guw4cjyRJUqdoqVJ0H7VA9JeImAz8Bnh12crMvLbBY5MkSQ0UzT46rLpaM6eoH/Ai8Bn+935FSe0ZJpIkaQ3l6bOylkJR/+LKs0d551Nqvdu0JElrOENRWUuhqDu155M095UZiiRJWsOFl5+VtBSKZmXmKR02EkmSpE7UUigyPkqS1IV5+qyspVC0c4eNQpIkdTjPnpUtNxRl5ryOHIgkSepYPuajbLk3b5QkSaqS1tynSJIkdUHOKSozFEmSVFGePSszFEmSVFHdvNC8xFAkSVJFWSkqc6K1JEkSVookSaosJ1qXGYokSaoo71NUZiiSJKmizERlhiJJkirKSlGZE60lSZKwUiRJUmVZKCozFEmSVFGeLiozFEmSVFFhqajEkChJkoSVIkmSKss6UZmhSJKkivKS/DJPn0mSVFHRzqVVx4joExFXR8TfIuKJiPhYRPSLiCkR8VTxs2/d9idGxLSIeDIiRtS1bx8RjxTrzo5iQlRErBMRVxXtUyNis7Z+H4YiSZIqKqJ9Syv9DLglM7cEPgw8AZwA3J6ZQ4Hbi/dExFbAGGBrYCRwbkR0L/ZzHjAeGFosI4v2ccD8zNwCOAs4o63fh6FIkiQ1RET0Aj4FXACQmW9l5kvAKOCSYrNLgN2L16OAiZn5ZmZOB6YBwyJiINArM+/JzAQubdJn2b6uBnaONl5WZyiSJKmiIqK9y/iIeKBuGd/kEO8DngcuioiHIuJXEbEeMCAzZwEUP/sX2w8Cnq3rP6NoG1S8btpe6pOZi4EFwIZt+T6caC1JUkW1tzKSmROACS1s0gPYDvh6Zk6NiJ9RnCpbjuYqPNlCe0t9VpqVIkmSKqq9laJWmAHMyMypxfurqYWkOcUpMYqfc+u237Su/2BgZtE+uJn2Up+I6AH0Buat5FcBGIokSaqsRl99lpmzgWcj4gNF087A48BkYGzRNha4oXg9GRhTXFE2hNqE6vuKU2wLI2J4MV/o4CZ9lu1rNHBHMe9opXn6TJIkNdLXgSsiYm3gaeAQakWZSRExDngG2BsgMx+LiEnUgtNi4IjMXFLs53DgYqAncHOxQG0S92URMY1ahWhMWwcabQxTDffyG0tXz4FJXdzLry/q7CFIlTW47zodejfFqx+e1a6/taM/PLBL3f3RSpEkSRXlHJoyQ5EkSRXVxtv5dFmGREmSJKwUSZJUWdaJygxFkiRVlGfPygxFkiRVVDdrRSWGIkmSKspKUZkTrSVJkrBSJElSZYWnz0oMRZIkVZSnz8oMRZIkVZQTrcsMRZIkVZSVojInWkuSJGGlSJKkyrJSVGYokiSporz6rMxQJElSRXUzE5U4p0iSJAkrRZIkVZanz8oMRZIkVZQTrcsMRZIkVZSVojJDkSRJFeVE6zInWkuSJGEo0nKc8t2T2GWnT7Dvnruukv3dNPl69tx1BHvuOoKbJl//jvU/+sH/41PDt18lx5LWFHPnzObYr43jkH1Hceh+e3DNVZe/Y5tn/jmdI//1QEZ+cnsmXXHxKjnuW2+9xaknHc9Bo7/IEYfuz+yZz5XWv/rqK+yz62c5+8zvr5LjafUV7fxfV2MoUrO+NGp3zj5vwkr3O2zcwcx8rvwP7IIFL3H+L3/BRZdfxcVXTOL8X/6Cl19e8Pb6xx97lIULX273mKU1Tffu3fnqUcdx0VU3cM6vLueGq6/in9P/Udpmg169OPLYE9h7/7Ervf/ZM5/j2MMPfUf7zZOvZf1evbjs6t+y134Hcf4vflpaf9F/nsOHt/U/Uqogon1LV2MoUrO2234HevXqU2qb8ewzfP3wr3DQmL34ypcP5J/Tn27Vvu790x/ZcfjH6d27D7169WbH4R/nnj/eDcCSJUs4+yc/4qhvfHNVfwRptbfhRhvz/i23AuBd663Hezcbwgtz55a26dtvQ7bcaht69HjnFNApN9/E1w7dn/EH7c1PTj+FJUuWtOq4f/rDnezyhd0A+L+f/hwPPjCVzATg7397nPnz5rH9sI+345NpTRHtXLqaDg9FEXFIRx9Tq8Zpp5zM8SecxGUTr+HoY4/njNNOaVW/uXPnMGCTTd5+33/AAObOnQPApIlX8KmdPs1GG/dvyJilNcXsmc8x7e9/44Pb/Eurtv+f6U9z53/fwtkTLmHCZb+he7du3H7rb1vV94Xn59B/wAAAuvfowXrrr8/LC15i6dKl/PJnZ3LY149t8+fQmqVbRLuWrqYzrj77HnBRcysiYjwwHuCn55zHIePGd+S41ILXXnuVRx5+iBOO/8bbbYveeguAyddfy8RfXwbAjGee4ZgjD6PHWmsx6N2D+NFPz4Hiv0DrBcHzc+dy+2238ssLLumYDyGtpl5/7TX+48Rj+dox32K99dZvVZ+HHpjKU08+wdcO2R+AN998gz59+wHw3W8fw+yZz7Fo0SLmzpnF+IP2BmDPfQ9g5Jd2b+5XEiKYfM1VDPv4/6H/gE2a2UDq+hoSiiLir8tbBQxYXr/MnABMAHj5jaXN/dqqkyxdmqy/wQb8etJ171i32+57stvuewK1OUUnn/ID3j1o0Nvr+w/YhD/ff9/b7+fOmcP2Owzjyb89zrPPPsOeu44A4I03XmePL43guptubfCnkVYfixcv4j9OPJadR3yRT376s63ul5ns8oXd+NevHf2Odaec8VOgVn364an/zk/Ou7C0fuP+A5g7Zw4b99+EJYsX8+orr9CrV28ef+RhHnn4QSZfM4nXX3+NxYsW0bPnu/jKEce05yNqNdb1aj3t06hK0QBgBDC/SXsAf2rQMdVA66+/Pu8eNJj/vu0WPrvLSDKTp/7+JO//wJYr7Dv845/g3LPPenty9dR7/sgRR3+D3r37cOsdf3h7u08N395ApErJTM487WTes9kQ9t7/4JXqu+0OO/Ld449mrzEH0rffhry8YAGvv/YqAwa+e4V9P/bJnbjtvyaz9b98mN//bgrbfnQYEcF3Tjn97W1uuekG/v63xwxEXZ2pqKRRoegmYP3M/EvTFRFxZ4OOqVXopG8fx58fuI+XXnqJL35uJ8YffiSnfv9HnH7a97jw/F+yePFiPjfi860KRb1792Hc+MMZu/8+AIw77Gv07t2nwZ9AWv09+vBDTLn5JoZsPvTtU1zjDj+KubNnAbDrnvsw78UXOPzLY3jt1VeJbt24ZuLlXDjxejYbsjmHHHYk3z76qyxdupQePXpw1PHfaVUo+sKue/CD732Hg0Z/kQ169ebfTv1hQz+nVl9d8bL69ohs9uRy5/P0mdQ5Xn59UWcPQaqswX3X6dCUMvUfC9r1t3bHzXt3qVTlYz4kSaqoLngBWbt4nyJJkiqqI+5TFBHdI+KhiLipeN8vIqZExFPFz751254YEdMi4smIGFHXvn1EPFKsOzuiFuciYp2IuKponxoRm7Xn+zAUSZJUVR1z98ajgSfq3p8A3J6ZQ4Hbi/dExFbAGGBrYCRwbkR0L/qcR+2WPUOLZWTRPg6Yn5lbAGcBZ7R6VM0wFEmSVFGNfvZZRAwGvgj8qq55FLDsBnWXALvXtU/MzDczczowDRgWEQOBXpl5T9YmQl/apM+yfV0N7LysitQWhiJJktQoPwW+BSytaxuQmbMAip/LHmkwCHi2brsZRdug4nXT9lKfzFwMLAA2bOtgDUWSJFVUex8IGxHjI+KBumX8/+47vgTMzcw/t3Y4zbRlC+0t9WkTrz6TJKmi2nvxWf2TKJrxCWC3iPgCsC7QKyIuB+ZExMDMnFWcGlv2FOQZwKZ1/QcDM4v2wc201/eZERE9gN7AvLZ+HitFkiRVVQMnWmfmiZk5ODM3ozaB+o7MPBCYDIwtNhsL3FC8ngyMKa4oG0JtQvV9xSm2hRExvJgvdHCTPsv2Nbo4hpUiSZK0cjrpjtanA5MiYhzwDLA3QGY+FhGTgMeBxcARmbmk6HM4cDHQE7i5WAAuAC6LiGnUKkRj2jMw72gtqcQ7Wkudp6PvaP3Q/yxs19/abd+7QZe6/aOVIkmSKso7WpcZiiRJqigzUZmhSJKkqjIVlXj1mSRJElaKJEmqrE66+my1ZSiSJKminGhdZiiSJKmizERlhiJJkqrKVFTiRGtJkiSsFEmSVFlOtC4zFEmSVFFOtC4zFEmSVFFmojLnFEmSJGGlSJKk6rJUVGIokiSpopxoXWYokiSpopxoXWYokiSposxEZU60liRJwkqRJEnVZamoxFAkSVJFOdG6zFAkSVJFOdG6zFAkSVJFmYnKnGgtSZKElSJJkqrLUlGJoUiSpIpyonWZoUiSpIpyonWZc4okSZKwUiRJUmVZKCozFEmSVFWmohJDkSRJFeVE6zJDkSRJFeVE6zInWkuSJGEokiSpsqKdywr3H7FpRPwuIp6IiMci4uiivV9ETImIp4qffev6nBgR0yLiyYgYUde+fUQ8Uqw7O6JW54qIdSLiqqJ9akRs1tbvw1AkSVJFRbRvaYXFwHGZ+UFgOHBERGwFnADcnplDgduL9xTrxgBbAyOBcyOie7Gv84DxwNBiGVm0jwPmZ+YWwFnAGW39PgxFkiRVVmNrRZk5KzMfLF4vBJ4ABgGjgEuKzS4Bdi9ejwImZuabmTkdmAYMi4iBQK/MvCczE7i0SZ9l+7oa2HlZFWllGYokSaqoDqgU1R0rNgO2BaYCAzJzFtSCE9C/2GwQ8GxdtxlF26DiddP2Up/MXAwsADZcudHVGIokSVKbRMT4iHigbhm/nO3WB64BjsnMl1vaZTNt2UJ7S31WmpfkS5JUUe29Ij8zJwATWjxGxFrUAtEVmXlt0TwnIgZm5qzi1Njcon0GsGld98HAzKJ9cDPt9X1mREQPoDcwry2fx0qRJEkV1ejTZ8XcnguAJzLzJ3WrJgNji9djgRvq2scUV5QNoTah+r7iFNvCiBhe7PPgJn2W7Ws0cEcx72ilWSmSJKmiOuCO1p8ADgIeiYi/FG3fAU4HJkXEOOAZYG+AzHwsIiYBj1O7cu2IzFxS9DscuBjoCdxcLFALXZdFxDRqFaIxbR1stDFMNdzLbyxdPQcmdXEvv76os4cgVdbgvut06D2mZy9Y1K6/tZv0XqtL3RPbSpEkSVXVpSJN+xmKJEmqKDNRmaFIkqSK8oGwZYYiSZIqqgMmWq9RvCRfkiQJK0WSJFWXhaISQ5EkSRVlJiozFEmSVFFOtC4zFEmSVFFOtC5zorUkSRJWiiRJqixPn5VZKZIkScJKkSRJlWWlqMxKkSRJElaKJEmqLK8+KzMUSZJUUZ4+KzMUSZJUUWaiMkORJElVZSoqcaK1JEkSVookSaosJ1qXGYokSaooJ1qXGYokSaooM1GZoUiSpKoyFZU40VqSJAkrRZIkVZYTrcsMRZIkVZQTrcsiMzt7DOqCImJ8Zk7o7HFIVePvntR2zilSo4zv7AFIFeXvntRGhiJJkiQMRZIkSYChSI3jnAapc/i7J7WRE60lSZKwUiRJkgQYirSKRcTIiHgyIqZFxAmdPR6pKiLiwoiYGxGPdvZYpDWVoUirTER0B34BfB7YCtgvIrbq3FFJlXExMLKzByGtyQxFWpWGAdMy8+nMfAuYCIzq5DFJlZCZdwHzOnsc0prMUKRVaRDwbN37GUWbJEmrPUORVqXmnqLj5Y2SpDWCoUir0gxg07r3g4GZnTQWSZJWiqFIq9L9wNCIGBIRawNjgMmdPCZJklrFUKRVJjMXA0cCtwJPAJMy87HOHZVUDRFxJXAP8IGImBER4zp7TNKaxjtaS5IkYaVIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkdRpImJJRPwlIh6NiN9ExLvasa+LI2J08fpXLT2INyJ2ioiPt+EY/4yIjVrb3mSbV1byWP8REd9c2TFKUnsYiqTO83pmfiQztwHeAr5avzIiurdlp5n5r5n5eAub7ASsdCiSpK7OUCStHv4AbFFUcX4XEb8GHomI7hHxo4i4PyL+GhGHAUTNORHxeET8Fui/bEcRcWdEfLR4PTIiHoyIhyPi9ojYjFr4+kZRpfpkRGwcEdcUx7g/Ij5R9N0wIm6LiIci4j9p/tl2JRFxfUT8OSIei4jxTdb9uBjL7RGxcdG2eUTcUvT5Q0Rs2cw+jyo+518jYmIbv19JWqEenT0AqeoiogfweeCWomkYsE1mTi+CxYLM3CEi1gH+GBG3AdsCHwD+BRgAPA5c2GS/GwPnA58q9tUvM+dFxC+BVzLzzGK7XwNnZebdEfEeanck/yBwMnB3Zp4SEV8ESiFnOQ4tjtETuD8irsnMF4H1gAcz87iI+G6x7yOBCcBXM/OpiNgROBf4TJN9ngAMycw3I6JPa75TSWoLQ5HUeXpGxF+K138ALqB2Wuu+zJxetO8CfGjZfCGgNzAU+BRwZWYuAWZGxB3N7H84cNeyfWXmvOWM47PAVhFvF4J6RcQGxTH2LPr+NiLmt+IzHRURexSvNy3G+iKwFLiqaL8cuDYi1i8+72/qjr1OM/v8K3BFRFwPXN+KMUhSmxiKpM7zemZ+pL6hCAev1jcBX8/MW5ts9wVgRc/oiVZsA7XT6B/LzNebGUurnwMUETtRC1gfy8zXIuJOYN3lbJ7FcV9q+h0044vUAtpuwL9HxNbFc/YkaZVyTpG0ersVODwi1gKIiPdHxHrAXcCYYs7RQODTzfS9B/i/ETGk6NuvaF8IbFC33W3UTmVRbPeR4uVdwAFF2+eBvisYa29gfhGItqRWqVqmG7Cs2rU/tdNyLwPTI2Lv4hgRER+u32FEdAM2zczfAd8C+gDrr2AcktQmVoqk1duvgM2AB6NWunke2B24jtrcm0eAvwO/b9oxM58v5iRdW4SLucDngBuBqyNiFPB14CjgFxHxV2r/JtxFbTL294ArI+LBYv/PrGCstwBfLfbzJHBv3bpXga0j4s/AAmDfov0A4LyI+DdgLWAi8HBdv+7A5RHRm1rl66zMfGkF45CkNonMVlfHJUmSuixPn0mSJGEokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgD4/53RAV9/WEy0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import seaborn as sns\n",
    "y_pred=np.array(y_pred)\n",
    "confusion_matrix_result=confusion_matrix(y,y_pred)\n",
    "plt.figure(figsize=(10,6))\n",
    "sns.heatmap(confusion_matrix_result,annot=True,cmap='Blues')\n",
    "plt.xlabel('Predicted labels')\n",
    "plt.ylabel('True labels')\n",
    "plt.show()\n",
    "#混淆矩阵将错误数据可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型可利用性评价\n",
    "&ensp;&ensp;&ensp;&ensp;对于神经网络的分类模型，本人采用**AUC**评价方法来检验神经网络分类是否合理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### AUC模型算理<br>\n",
    "&ensp;&ensp;&ensp;&ensp;AUC是现在分类模型，特别是二分类模型使用的主要离线评测指标之一。相比于准确率、召回率、F1等指标，AUC有一个独特的优势，就是不关注具体得分，只关注排序结果，这使得它特别适用于排序问题的效果评估，例如推荐排序的评估。AUC这个指标有两种解释方法，一种是传统的“曲线下面积”解释，另一种是关于排序能力的解释。例如0.7的AUC，其含义可以大概理解为：给定一个正样本和一个负样本，在70%的情况下，模型对正样本的打分高于对负样本的打分。可以看出在这个解释下，我们关心的只有正负样本之间的分数高低，而具体的分值则无关紧要。<br>\n",
    "&ensp;&ensp;&ensp;&ensp;AUC是在ROC的基础上建立起来的，因此，在了解什么是AUC之前需要知道什么是ROC。<br>\n",
    "##### ROC曲线的实例<br>\n",
    "![](https://imgconvert.csdnimg.cn/aHR0cDovL3VwbG9hZC1pbWFnZXMuamlhbnNodS5pby91cGxvYWRfaW1hZ2VzLzUwODI4LWM0ZGZmZjYxZTNhZDU4ZmEucG5n?x-oss-process=image/format,png)\n",
    "正如我们在这个ROC曲线的示例图中看到的那样，ROC曲线的横坐标为false positive rate（FPR），纵坐标为true positive rate（TPR）（也就是recall）。\n",
    "接下来我们考虑ROC曲线图中的四个点和一条线。\n",
    "\n",
    "第一个点，(0,1)，即FPR=0, TPR=1，这意味着FN（false negative）=0，并且FP（false positive）=0。Wow，这是一个完美的分类器，它将所有的样本都正确分类。\n",
    "\n",
    "第二个点，(1,0)，即FPR=1，TPR=0，类似地分析可以发现这是一个最糟糕的分类器，因为它成功避开了所有的正确答案。\n",
    "\n",
    "第三个点，(0,0)，即FPR=TPR=0，即FP（false positive）=TP（true positive）=0，可以发现该分类器预测所有的样本都为负样本（negative）。\n",
    "\n",
    "第四个点（1,1），分类器实际上预测所有的样本都为正样本。经过以上的分析，我们可以断言，ROC曲线越接近左上角，该分类器的性能越好。\n",
    "\n",
    "下面考虑ROC曲线图中的虚线y=x上的点。这条对角线上的点其实表示的是一个采用随机猜测策略的分类器的结果(FP = TN, TP = FN，这样FP+TP = TN + FN，即Y = N，也就是随机猜测了)，例如(0.5,0.5)，表示该分类器随机对于一半的样本猜测其为正样本，另外一半的样本为负样本。\n",
    "\n",
    "##### 如何画ROC曲线<br>\n",
    "通过调整模型预测的阈值可以得到不同的点，将这些点可以连成一条曲线，这条曲线叫做接受者工作特征曲线(Receiver Operating Characteristic Curve，简称ROC曲线）。\n",
    "\n",
    "假如我们已经得到了所有样本的概率输出（属于正样本的概率），我们根据每个测试样本属于正样本的概率值从大到小排序。下图是一个示例，图中共有20个测试样本，“Class”一栏表示每个测试样本真正的标签（p表示正样本，n表示负样本），“Score”表示每个测试样本属于正样本的概率。\n",
    "![](https://img-blog.csdnimg.cn/20200210115850555.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly95b3V6aXBpLmJsb2cuY3Nkbi5uZXQ=,size_16,color_FFFFFF,t_70)\n",
    "接下来，我们从高到低，依次将“Score”值作为阈值threshold，当测试样本属于正样本的概率大于或等于这个threshold时，我们认为它为正样本，否则为负样本。举例来说，对于图中的第4个样本，其“Score”值为0.6，那么样本1，2，3，4都被认为是正样本，因为它们的“Score”值都大于等于0.6，而其他样本则都认为是负样本。每次选取一个不同的threshold，我们就可以得到一组FPR和TPR，即ROC曲线上的一点。这样一来，我们一共得到了20组FPR和TPR的值，将它们画在ROC曲线的结果如下图：\n",
    "![](https://img-blog.csdnimg.cn/20200210115901696.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly95b3V6aXBpLmJsb2cuY3Nkbi5uZXQ=,size_16,color_FFFFFF,t_70)\n",
    "当我们将threshold设置为1和0时，分别可以得到ROC曲线上的(0,0)和(1,1)两个点。将这些(FPR,TPR)对连接起来，就得到了ROC曲线。当threshold取值越多，ROC曲线越平滑。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;AUC（Area Under Curve）被定义为ROC曲线下的面积，显然这个面积的数值不会大于1。又由于ROC曲线一般都处于y=x这条直线的上方，所以AUC的取值范围在0.5和1之间。使用AUC值作为评价标准是因为很多时候ROC曲线并不能清晰的说明哪个分类器的效果更好，而作为一个数值，对应AUC更大的分类器效果更好。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;更多算理讲解，见<http://blog.csdn.net/pipisorry/article/details/51788927>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 利用AUC方法评价神经网络模型预测的可用性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABFiklEQVR4nO3dd3gU5fbA8e9JQkiA0EJROkgJ5QJKBylSpIjtZ8Fy8eLVK6HZUPFiwy4WEKWJqHht2AFBRbEAotIkdAREhCC9hJIEUs7vjxnCElI2kM3uJufzPPvs9Dk7uztn3ndm3hFVxRhjjMlOiL8DMMYYE9gsURhjjMmRJQpjjDE5skRhjDEmR5YojDHG5MgShTHGmBxZoigkRGStiHTxdxz+JiKTReSRAl7nNBF5qiDX6SsicrOIfHOW8xba36CIqIjU9Xcc/iJ2H0X+E5GtQGUgDTgKfA0MVdWj/oyrsBGRAcDtqnqxn+OYBsSr6sN+jmMUUFdV/1kA65pGAHzmgiIiCtRT1c3+jsUfrEThO5eraimgOXAh8F//hpN3IhJWFNftT7bNTUBSVXvl8wvYCnT36H8emOPR3xb4GTgErAS6eIwrD7wF/A0cBGZ4jOsLxLnz/Qw0zbxOoAqQBJT3GHchsA8o5vb/G1jvLn8uUNNjWgWGAJuAP7P5fFcAa904fgQaZorjv8A6d/lvARF5+AwjgFXAcSAMeBD4AzjiLvNqd9qGQDKnSm2H3OHTgKfc7i5APDAc2APsBG71WF808AVwGFgKPAX8lMP3erHH97YdGOCxzgnAHDfOxcAFHvONc6c/DCwHOnqMGwV8Arzrjr8daA384q5nJzAeCPeYpzHwLXAA2A2MBHoBJ4AUd3usdKctA7zhLmeH+xlD3XEDgEXAWHdZT7nDfnLHiztuD5Dgfi9NgDvc9Zxw1/VF5t89EOrGdfK7Ww5Uz2a7Zvl/ANrj/G6ru/3N3Gli3P4sfxtZfLZDwBZ3eQPc72IP8C+P6acBk93tegSYz5n/i7pud3HgRWCbu/0nA5H+3u/4dJ/m7wAK4yvTH6YasBoY5/ZXBfYDfXBKdD3c/oru+DnAh0A5oBjQ2R1+kfvjbuP+Cf/lrqd4Fuv8HviPRzwvAJPd7quAzTg72jDgYeBnj2nV/bOUz+rHD9QHjrlxFwMecJcX7hHHGqC6u4xFnNpxe/MZ4tx5I91h1+EkvxCgn7vu891xA8i0Y+fMRJEKPOHG2gdIBMq546e7rxJAI5wdSJaJAqiBswO50V1WNNDcY50HcHbwYcB7wHSPef/pTh+Gk7R24SZPnESR4n4vIUAk0AJn5xkG1MJJ6ne700fh7PSHAxFufxuPZb2bKe4ZwGtASaASsAQY6LH9UoFh7roiOT1R9MTZwZfFSRoNPbZ9xnbO5nd/P87vvoE7bzMgOovtmtv/4Wmc33MkTqIa6jFvbr+NVOBWnN/aUzg79gk4O/pL3e+zlMfnOQJ0csePw+O3wOmJ4mVgFs7vOwrnYONZf+93fLpP83cAhfHl/mGOuj88Bb4DyrrjRgDvZJp+Ls5O83wgHXdHlmmaScCTmYb9zqlE4vknvR343u0WnB1gJ7f/K+A2j2WE4Ow8a7r9CnTN4bM9AnyUaf4dnDoK3ArEeozvA/yRh8/w71y2bRxwpds9gNwTRRIQ5jF+D85OOBRnB93AY1y2JQqcUtLn2YybBkzN9Jk35PAZDgLN3O5RwIJcPvPdJ9eNk6hWZDPdKDwSBc55suN4JHx3/h88tt+2TMvI2KZAV2Cju71CstvOmX73J3+Dv5/8nnL5bNn+H9zuYjjJajXOuT7Jw29jk8e4f+D8tit7DNvP6cneM7mXwimtnizNKFAX5/90jNNLjO3IpvRdWF52jsJ3rlLVKJydVQxQwR1eE7hORA6dfOFUaZyPcyR9QFUPZrG8msDwTPNVxzmiyuwToJ2IVME5QlJgocdyxnks4wDOj7+qx/zbc/hcVYC/Tvaoaro7fXbz/+URozef4bR1i8gtIhLnMX0TTm1Lb+xX1VSP/kScnUBFnKNoz/Xl9Lmr41RzZGdXFusAQESGi8h6EUlwP0MZTv8MmT9zfRGZLSK7ROQw8IzH9LnF4akmzo52p8f2ew2nZJHluj2p6vc41V4TgN0iMkVESnu5bm/jzOn/gKqm4OzEmwAvqbtnBq9+G7s9upPc5WUeVsqjP2NbqHPhyQHO/H9VxCmBLvdY79fu8ELLEoWPqep8nB/6i+6g7ThHUGU9XiVV9Tl3XHkRKZvForYDT2ear4SqfpDFOg8B3wDXAzcBH3j8wbbjVD14LidSVX/2XEQOH+lvnD83ACIiODuFHR7TVPforuHO4+1n8NwR1AReB4biVFuUxanWEi/izM1enKqJatnEndl24IK8rkREOuIcNV+PU1Isi1PfLx6TZf4ck4ANOFfZlMap6z85fU5xZF7OdpwSRQWP7V1aVRvnMM/pC1R9RVVb4JwXqY9TpZTrfLnEmXm67P4PiEhV4DGcc10viUhxd3huv42zkfH9i0gpnKqlvzNNsw8nwTT2iLeMOheuFFqWKArGy0APEWmOc9LychHpKSKhIhIhIl1EpJqq7sSpGpooIuVEpJiIdHKX8ToQKyJtxFFSRC4Tkahs1vk+cAtwjdt90mTgvyLSGEBEyojIdXn4LB8Bl4lINxEphlNXfhznZORJQ0SkmoiUx9nJfXiWn6Ekzg5prxvrrThHjSftBqqJSHge4gdAVdOAz4BRIlJCRGJwtld23gO6i8j1IhImItHu95mbKJyEtBcIE5FHgdyOyqNwTmwfdeMa5DFuNnCeiNwtIsVFJEpE2rjjdgO1RCTE/Yw7cQ4YXhKR0iISIiIXiEhnL+JGRFq531UxnOqWkxcPnFxXnRxmnwo8KSL13O+6qYhEZzFdtv8H9yBkGs7J+Ntwzs086c6X22/jbPQRkYvd39OTwGJVPa3E5ZagXwfGikgld91VRaTnOa47oFmiKACquhf4H/CI+8O7EmcHuhfniOp+Tn0X/XHqzjfg1Kff7S5jGfAfnKqAgzgnkAfksNpZQD1gt6qu9Ijlc2A0MN2t1lgD9M7DZ/kd5+TsqzhHV5fjXAp8wmOy93F2UFvc11Nn8xlUdR3wEs4VQLtx6pkXeUzyPc7VV7tEZJ+3n8HDUJxqoF3AO8AHOEkvq1i24Zx7GI5TJRGHc4I2N3Nxkv9GnGq4ZHKu4gK4D6ckeARnp3Qy0aKqR3BO+F7uxr0JuMQd/bH7vl9EfnO7bwHCOXUV2ie41TpeKO2u/6Ab+35OlYzfABq51S8zsph3DM5BxTc4Se8NnBPSp8nl/3AnznmWR9wS8a3ArSLS0Yvfxtl4H6f0cgDngoKbs5luBM5v91f3PzQP56R9oWU33Jl8Jc7Nhrer6jx/x5JXIjIaOE9V/+XvWEzBkiJ2A2FeWYnCFFkiEuNWiYiItMap3vjc33EZE2jsTkxTlEXhVDdVwanmewmY6deIjAlAVvVkjDEmR1b1ZIwxJkdBV/VUoUIFrVWrlr/DMMaYoLJ8+fJ9qnpWNwYGXaKoVasWy5Yt83cYxhgTVETkr9ynyppVPRljjMmRJQpjjDE5skRhjDEmR5YojDHG5MgShTHGmBxZojDGGJMjnyUKEXlTRPaIyJpsxouIvCIim0VklYhc5KtYjDHGnD1f3kcxDac56f9lM743TjPY9XCeoTzJfTfGO6qg6ae/SD9zmKZD+smH3OmpebPqzm28ekyXXXdeptVM6z1tGJnGeXzuMzdGLtPk0lRPjk35+HDenMbn2ryQj+bNdf7g2x4nTqTnMm/OfJYoVHWBiNTKYZIrgf+57cz/KiJlReR892ErJlClJEHSvlOvlCOQmuy+kiDNfc88LOXYqVfaCUg/4b6nONOkp7o79TSPHbxHd3oakO4s0xjjtXEL2zB18blV2PjzzuyqnP4Al3h32BmJQkTuAO4AqFGjRoEEV+SkJMKhP2DvSkjcDcd2ebzvgeQDkLQfUhP9HenpJOT0FyFnDpMQQCBpL5Q8j4ynZYqc6kbcfrIef0Z3VvPkYdps58s8Ho/pPHuzeuJnbtPk9pTQHMZnub5CPG+u8wfPZ2p2tDTrvqiUxbTe82eiyOoTZ1l2UtUpwBSAli1bWnO35yJpP+xfDwfc18nuw17e3R9SDEpUhMiKEBkN4aUhLBJCIyAswukOizi9PzQCipU89QotDqHhzrJCw53pQsIz7dxDs9jhu8NCQt3uc3k8sjGF0/btCcyevZFBg1oB0OUa2HzLQerUefysl+nPRBHP6Q+zr8aZDzI35+LoTvh7Efz9K+xfC7uXOdVFWQkJg1LVoMI/oGwdKHGec/RdsjKUqAQR0RBRHsKjbAdtTABKTU3nlVcW8+ijP3DsWApNmlSiY8eaANSuXe6clu3PRDELGCoi03FOYifY+YlzcHgb7FgIBzbC4a1Ogjj0x5nTFSsJ5WOgfEOIbui8l28IZS+A0GIFHrYx5twtXhzPwIGzWblyNwDXXNOQOnXOLTl48lmiEJEPgC5ABRGJx3loeTEAVZ0MfInzsPrNQCLOg9ONt1Th719g02fw1zewb/WZ0xQrBVXaQZX2UKEJnNcKompYicCYQuLgwSRGjvyO115bjirUqlWW8eN7c9ll9fN1Pb686unGXMYrMMRX6y+00lOd5LB8DOxcfGp4sVJQvTNUbA6la0GFxk5iCAm6luSNMV56/PH5TJ68nLCwEO67rx2PPNKZEiXyv2bA9iLB4vhhWPMG/Dbu1InniPLQ5N9QswdU6wxhxf0bozHG51JT0wkLc+6VfvjhTvz55yGefrorTZqc25VNObFEEehOHIGlz8Nvr8CJw86wcvXgonug8S3OOQdjTKGXnJzK6NE/MWPG7yxefDvh4aFUqFCCmTNv8Pm6LVEEqrQUpwTx82POfQzglBpa3AsX9HXvDTDGFAXffbeFQYPmsGnTAQDmzt3M5Zc3KLD1W6IIJIl7Ye002PET/P3zqUtZq7SHTi9A1fZ+Dc8YU7B27z7K8OHf8N57zsUqDRtWYNKky+jcuVaBxmGJIhAk7oVlL8KK8aff+VyuHlz8DNS7xq5UMqaIeffdVQwb9hWHDiUTERHGo492Yvjw9oSHhxZ4LJYo/Clxn5Mg4sY7bSAB1O4DMTfA+e2cexssQRhTJKWnK4cOJdOrV10mTOiTr/dF5JUlCn9I3AfLX4IVr55KEHUug3aPOZe0GmOKnKNHT/DLL9vp0eMCAPr3b0qVKlF061Yb8fMBoyWKgpS417n/wTNB1O7jJIjzW/s3NmOM38yYsYFhw75i795jrFkzmLp1yyMidO9ex9+hAZYoCsaRHfDL486J6vQUZ1jt3tBulCUIY4qwv/46xJ13fs2sWb8D0LJlFY4fT81lroJnicKXkg/CktGwYpzzHAUJgTp9oe3DcL49o8mYoiolJY2XX/6VUaPmk5iYQlRUOM88041Bg1oSGhp4l75bovCFlGPwyxNOFVNqkjOs3jVw8dNQvuCufTbGBKY77/yKyZOXA3D99Y0ZO7YnVapE+Tmq7FmiyG9H4uHDzpCwxemv2QM6PGVVTMaYDHff3Zb58/9izJie9OpV19/h5MoSRX5RhfXvwvfD4HgClKsPvf9nVUzGFHGqyrvvruLLLzfz/vv/h4jQoEEF1qwZTEhIcFz+bokiPyQfhDk3wdavnf5qneHyj5wH/hhjiqzff9/HoEFz+OGHrYBzyWufPvUAgiZJgCWKc7d/Hcy4Eg5tdp4Cd/FT0HSg3ShnTBGWlJTCs8/+xOjRizhxIo3o6EheeulSevcO/GqmrFiiOFtpJ2DJc7D0BUg56jwH4qoZULqmvyMzxvjRvHlbiI2dzR9/HATgttsuZPTo7kRHl/BzZGfPEsXZ2LsK5t8Hf33r9De4AXq+AcWC94dgjMkfP/+8nT/+OEjjxhWZPLkvF19cw98hnTNLFHm18jWYF+t0R0RD3+lQo5tVNRlTRKWlpbN58wEaNKgAwIgRHahQoQS3336RXxrw84XAu7MjkK1+41SSaBYL/ZdDze6WJIwpolas2En79m9y8cVvceCAc89U8eJhDB7cqtAkCbBE4b3t8+HbO5zuzi9B90l2PsKYIurIkePcc8/XtGz5OkuW7KB48VD++OOAv8PyGat68kbiXvjyJtB0aDUCWt7r74iMMX6gqnz22Xruuutrduw4QkiIcM89bXn88S5ERRXeZ9ZboshN6nGY9X9w9G+o0sG5/NUYUyTdfffXvPLKEgBatarCa6/15cILz/dzVL5nVU85UYWv+juPJi1VFfp+CCGWW40pqq6+uiFlyhRnwoQ+/PLLbUUiSYCVKHK27EXY+DEUKwlXz4aoqv6OyBhTgH76aRs//PAnjzzSGYAuXWqxbds9lC5deKuZsmKJIjuHt8Oih53uPu9DpeZ+DccYU3D2709kxIh5vPHGCgC6datD+/bVAYpckgBLFFlThR/ucu6+btAP6l7h74iMMQVAVfnf/1Zy333fsm9fIsWKhfDggxdz4YXn+Ts0v7JEkVnSfvjmP7D5cyhWCjo97++IjDEFYP36vQwaNIf58/8C4JJLajFx4mXExFTwc2T+Z4kis/nDnSQBcMk4KB38t98bY3I3ZswvzJ//FxUrlmDMmJ7cfPM/ELuZFrBEcbqdi2Ht284jS//vS6jV098RGWN8KCEhmTJlIgB49tnulCwZzqOPdqZ8+Ug/RxZY7PLYkxK2wsyrnO4Wwy1JGFOI/f33Efr1+4S2bd/gxIk0ACpUKMHLL/eyJJEFSxQA6WnwxXVwbBdUvwQ6POnviIwxPpCWls6rry4mJmY8H320lm3bEvjtt53+DivgWdUTwJo3YfcyKFUNrvwcwore5W/GFHbLl//NwIGzWb7cSQxXXNGAV1/tTY0aZfwcWeDzaYlCRHqJyO8isllEHsxifBkR+UJEVorIWhG51ZfxZCklERY94nR3fhGK24/GmMJm1Kgfad16KsuX76R69dLMmNGPmTNvsCThJZ+VKEQkFJgA9ADigaUiMktV13lMNgRYp6qXi0hF4HcReU9VT/gqrjOseg0Sd0PlltDg+gJbrTGm4NSpUw4RGD68HaNGdaFUqXB/hxRUfFn11BrYrKpbAERkOnAl4JkoFIgS5xq0UsABINWHMZ0u6QD86p6PaPeoPVfCmEJiy5aDLF26g379mgDQv39T2rSpmvFwIZM3vkwUVYHtHv3xQJtM04wHZgF/A1FAP1VNz7wgEbkDuAOgRo18vK9h+RhIPgg1ukKdvvm3XGOMX5w4kcaLL/7Mk08uQFVp0aIKdeuWR0QsSZwDX56jyOrwXDP19wTigCpAc2C8iJQ+YybVKaraUlVbVqxYMX+iS9oPK15xuts/aaUJY4LcggV/0bz5ZB566HuSk1O59tpGRbJdJl/wZYkiHqju0V8Np+Tg6VbgOVVVYLOI/AnEAEt8GJdj+Rg4cQRqXgpV2/t8dcYY39i3L5H77/+WadPiAKhXrzyTJl1Gt251/BtYIeLLRLEUqCcitYEdwA3ATZmm2QZ0AxaKSGWgAbDFhzE5kvbDbydLE4/7fHXGGN+JjZ3Np5+up3jxUEaO7MgDD3QgIsKu/M9PPtuaqpoqIkOBuUAo8KaqrhWRWHf8ZOBJYJqIrMapqhqhqvt8FVOGJaMh5SjU6gVV2vp8dcaY/JWeroSEONXFTz/dlaSkVF5+uSf16kX7ObLCSZxan+DRsmVLXbZs2dkv4MgOeLMupCbDP5dB5Rb5F5wxxqcSE1N48sn5xMXt5ssvb7JG+/JARJarasuzmbfolc9+GukkiXrXWJIwJojMmbORoUO/YuvWQ4jAkiU7aNOmmr/DKhKKVqKIXwDr/gdhEXDxM/6Oxhjjhfj4w9x119d89tl6AJo1q8zkyX0tSRSgopMoNB1+uMfpbvUglK/v33iMMbmaOHEpI0bM4+jRE5QsWYwnn7yEYcPaEBZm7ZkWpKKTKP7+Ffb8BiXPg1b3+zsaY4wX9u1L5OjRE1x9dQzjxvWienVrm8kfik6i2Pix8x5zIxQr4d9YjDFZOnQomQ0b9tG2rVOtNGJEB1q3rkqvXnX9HFnRVjTKb0n7Ye1bTnfMjf6NxRhzBlVl+vQ1NGw4gSuu+IADB5IAKF48zJJEACgaiWLpC3A8AWp0h/Na+TsaY4yHzZsP0KvXe9x446fs2nWUevWiSUhI9ndYxkPhr3pKTztVmujwhH9jMcZkOH48leefX8TTTy/k+PE0ypWL4Pnne/Dvf1+YcTOdCQxeJwoRKamqx3wZjE/sXQWJeyCqOpxvd2EbEyj69fuEmTN/B+CWW5rxwgs9qFSppJ+jMlnJtepJRNqLyDpgvdvfTEQm+jyy/PL7dOe9dm9rIdaYAHL33W2JianA99/fwttvX2VJIoB5c45iLE5z4PsBVHUl0MmXQeUbTYf17zndDfv7NxZjirD0dGXq1N8YPnxuxrAuXWqxZs0gLrmkth8jM97wqupJVbdnalMlzTfh5LPt8+HoDihdy5oSN8ZPVq/eTWzsHH7+2XmO2S23NKNZs/MACA0tGtfTBDtvEsV2EWkPqIiEA3fiVkMFvLXTnPeGN4PYD9KYgnTs2Akef3w+Y8b8Qlqact55pXj55Z40bVrZ36GZPPImUcQC43AebRoPfAMM9mVQ+WLfWlj3DiDQ6BZ/R2NMkfLFF78zdOhXbNuWgAgMGdKKp5/uSpkyEf4OzZwFbxJFA1W92XOAiHQAFvkmpHyy6VNAofG/rF0nYwrYjBkb2LYtgQsvPI/XXutLq1ZV/R2SOQfeJIpXgYu8GBZY1r/rvDe4wb9xGFMEpKams2PHYWrWLAvA6NE9uPDC84mNbWkN+BUC2SYKEWkHtAcqisi9HqNK4zyxLnBtnw8HN0FEOajZw9/RGFOo/fprPLGxszl+PI2VK2MJDw+lQoUSDB3a2t+hmXySU6oPB0rhJJMoj9dh4Frfh3YO1r7tvFe/BEICO6cZE6wOHkxi0KDZtG//BitX7iY5OZWtWw/5OyzjA9mWKFR1PjBfRKap6l8FGNO5SU+DLbOd7jYP+zcWYwohVeWDD9Zwzz1z2bPnGGFhIdx/f3sefrgTJUoU83d4xge8OUeRKCIvAI2BjEsWVLWrz6I6F/vWQNJeKF0TKjX3dzTGFDo33/wZH3ywBoCOHWswadJlNG5cyc9RGV/y5izTe8AGoDbwOLAVWOrDmM7NrsXOe5X21mSHMT7Qq1ddoqMjefPNK/jxxwGWJIoAb0oU0ar6hojc5VEdNd/XgZ217W5o1gCgMfli3rwt/PHHAQYObAlA//5N6du3PuXLR/o5MlNQvEkUKe77ThG5DPgbCNynmu/81Xmv3sWvYRgT7HbvPsq9937D+++vpnjxULp3r8MFF5RHRCxJFDHeJIqnRKQMMBzn/onSwN2+DOqsJR2AhC0QFgnRjfwdjTFBKT1dmTJlOQ8+OI+EhONERITx6KOd7HnVRViuiUJV3UuISAAugYw7swPP7uXOe8VmEFL4n8lkTH5buXIXAwfOZvHiHQD07l2X8eP7UKdOOT9HZvwppxvuQoHrcdp4+lpV14hIX2AkEAlcWDAh5sHOX5z389v4Nw5jgtQDD8xj8eIdVKkSxbhxvbjmmoaIXRRS5OV02P0GUB1YArwiIn8B7YAHVXVGAcSWd7t/c94rt/BvHMYECVUlMTGFkiXDAXjllV5MnryMxx+/hNKli/s5OhMockoULYGmqpouIhHAPqCuqu4qmNDOwm73ql274smYXP311yGGDfuKY8dSmDevPyJCgwYVGDu2l79DMwEmp0RxQlXTAVQ1WUQ2BnSSOLYLjv4N4aWh7AX+jsaYgJWSksbYsb/y+OPzSUxMISoqnE2bDlC/frS/QzMBKqdEESMiq9xuAS5w+wVQVW3q8+jyYs8K573ShfaQImOysWjRNmJj57BmzR4A+vVrzJgxPalSJcrPkZlAllOiaFhgUeSHjPMTgd36uTH+MmzYl4wf71TP1qlTjgkT+tCrV10/R2WCQU6NAgZPQ4AAe9xEUSnwLsYyJhBUrFiSYsVCGDGiAyNHdiQy0hrwM97xaR2NiPQSkd9FZLOIPJjNNF1EJE5E1p5T0yAnSxSVrERhDMCGDfv45ps/MvpHjOjAqlWDePLJrpYkTJ747K409z6MCUAPnGdtLxWRWaq6zmOassBEoJeqbhORs2tdLPkgHN7q3JFdvsE5x25MMEtKSuGZZxYyevQiypaNYMOGoZQvH0nx4mHExFTwd3gmCHmVKEQkEqihqr/nYdmtgc2qusVdxnTgSmCdxzQ3AZ+p6jYAVd2Th+WfcvJEtt2RbYq4b775g8GD5/DHHwcBuOKKBtaIsjlnuVY9icjlQBzwtdvfXERmebHsqsB2j/54d5in+kA5EflRRJaLyC1eRZ2ZVTuZIm7nziPccMMn9Oz5Ln/8cZDGjSuycOGtTJ16BeXKWQN+5tx4c/g9Cqd08COAqsaJSC0v5svqOEazWH8LoBtOsyC/iMivqrrxtAWJ3AHcAVCjRo0zl7o3znm3E9mmiPq///uIX3+NJzIyjFGjunDPPW0pVsweA2zyhzcns1NVNeEslh2P0wTISdVwmijPPM3XqnpMVfcBC4BmmRekqlNUtaWqtqxYseKZazqwwXkvH3MWYRoTnFRPHXc991w3+vatz7p1Q3jggQ6WJEy+8iZRrBGRm4BQEaknIq8CP3sx31KgnojUFpFw4AYgc5XVTKCjiISJSAmgDbA+D/GDqiUKU6QcOXKce+75moEDZ2cM69y5Fl98cSO1apX1X2Cm0PImUQzDeV72ceB9nObG785tJlVNBYYCc3F2/h+p6loRiRWRWHea9TjnPlbhND44VVXX5OkTJO2HlGNO0x0l7IoOU3ipKp9+uo6GDSfw8suLeeutOLZuPeTvsEwR4M05igaq+hDwUF4XrqpfAl9mGjY5U/8LwAt5XXaGI+59gaVrnvUijAl0f/55kKFDv+LLLzcB0Lp1VSZPvsxKEKZAeJMoxojI+cDHwHRVXevjmPLmsCUKU3ipKs8/v4jHH59PUlIqZcoU59lnu3HHHS0IDbU2zUzB8OYJd5eIyHk4DzGaIiKlgQ9V9SmfR+cNSxSmEBMRNm7cT1JSKjfe2IQxY3py3nml/B2WKWK8OiRR1V2q+goQi3NPxaO+DCpPLFGYQmbfvsSM1l0BRo/uwTff/JP337/GkoTxC29uuGsoIqNEZA0wHueKp2o+j8xblihMIaGqTJsWR0zMeK677mNOnEgDoEKFEvToYc9YMf7jzTmKt4APgEtVNfN9EP5nicIUAuvX7yU2dg4LFji/52bNzuPgwSQqV7YShPE/b85RBPZzRS1RmCCWmJjC008v4IUXfiYlJZ2KFUswZkxPbr75H4g10mQCRLaJQkQ+UtXrRWQ1pze9EThPuDtxFJIPQGhxKHF2Dc8a4y+qSteub7N48Q4ABg5swbPPdrO2mUzAyalEcZf73rcgAjkrR+Kd96jq9vhTE3REhMGDW5GYmMJrr/WlXbvquc9kjB9ku3dV1Z1u52BV/cvzBQwumPBycfRkogicc+vGZCctLZ1XX13MmDG/ZAzr378py5ffYUnCBDRvDsN7ZDGsd34HclZOlihKWaIwgW3Zsr9p02Yqd975NSNHfsfffx8BnFKFNeBnAl1O5ygG4ZQc6ojIKo9RUcAiXwfmFStRmACXkJDMww9/z4QJS1GF6tVL8+qrvalSJcrfoRnjtZzOUbwPfAU8C3g+7/qIqh7waVTeshKFCVCqyscfr+Puu79m586jhIYK99zTlsce60KpUuH+Ds+YPMkpUaiqbhWRIZlHiEj5gEgWVqIwAey115azc+dR2ratxuTJl9Gs2Xn+DsmYs5JbiaIvsBzn8ljPi7oVqOPDuLxzxBKFCRzHj6dy6FAylSuXQkSYOLEPP/64lf/8pwUhIXZPhAle2SYKVe3rvtcuuHDyyKqeTICYP38rsbFzqFIlinnz+iMiNGhQgQYN7BkpJvh509ZTBxEp6Xb/U0TGiEgWD64uYClJkLwfQopBiSwej2pMAdi79xgDBsygS5e32bBhH9u3J7B79zF/h2VMvvLm8thJQKKINAMeAP4C3vFpVN446tzNSqmqdrOdKXDp6cobb/xGTMwE3n57JcWLh/L4411YtWqQtfBqCh1vGgVMVVUVkSuBcar6hoj8y9eB5cpOZBs/UVV69nyXefO2ANC9ex0mTuxDvXrRfo7MGN/wJlEcEZH/Av2BjiISChTzbVhesPMTxk9EhI4da7B69W7Gju3JDTc0sQb8TKHmTZ1NP+A48G9V3QVU5VyecZ1fMhJFVf/GYYqEOXM2MmPGhoz+ESM6sGHDUG680Vp5NYWfN82M7xKR94BWItIXWKKq//N9aLlI3OW8l6ri3zhMoRYff5i77vqazz5bT4UKJejUqSbly0dSvHgYxYt7UyA3Jvh5c9XT9cAS4Dqc52YvFpFrfR1Yro65iaKk3cRk8l9qajpjx/5Cw4YT+Oyz9ZQsWYyRIy+mdOni/g7NmALnzSHRQ0ArVd0DICIVgXnAJ74MLFeWKIyPLFmyg4EDZxMX5/zGrr46hnHjelG9ehk/R2aMf3iTKEJOJgnXfrw7t+FbliiMD6SnK7feOpN16/ZSo0YZxo/vzeWXN/B3WMb4lTeJ4msRmYvz3GxwTm5/6buQvHTyHEUJSxTm3Kgqx4+nERERRkiIMGFCH776ahOPPtqZkiWtAT9jvDmZfb+I/B9wMU57T1NU9XOfR5aTlCQ4nuDclR1Rzq+hmOC2efMBBg+eQ/XqpXnjjSsB6NKlFl261PJvYMYEkJyeR1EPeBG4AFgN3KeqOwoqsBwl7nbeS54HdmmiOQvHj6cyevQinnlmIcePp1G+fCTPP59IdHQJf4dmTMDJ6VzDm8Bs4BqcFmRfLZCIvGHnJ8w5+P77P2nadDKPPfYjx4+n8a9/NWPDhiGWJIzJRk5VT1Gq+rrb/buI/FYQAXnlmJ2fMHmXlpbOrbfO5J13nAc2NmgQzeTJfa2ayZhc5JQoIkTkQk49hyLSs19V/Zc4Eq1EYfIuNDSEsLAQIiLCePjhjtx3X3u7ac4YL+T0L9kJjPHo3+XRr0BXXwWVK6t6Ml5avXo3ycmptGrlNPXywgs9eOihjlxwQXk/R2ZM8MjpwUWXFGQgeXJsp/NuicJk49ixE4wa9SNjx/5KvXrRrFwZS3h4KNHRJexchDF5FJzl7oMbnffStfwahglMs2b9zrBhX7FtWwIi0L17bVJS0ggPD/V3aMYEJZ/eYS0ivUTkdxHZLCIP5jBdKxFJ87oNqSPbnfeyF+RLnKZw2LYtgauums6VV05n27YELrrofJYs+Q+vvtrHbpwz5hz4rEThPrdiAtADiAeWisgsVV2XxXSjgbleLzxpn/MeaY9ANY60tHS6dJnGn38eIioqnKee6srgwa0IC/N/azPGBLtcE4U4je3fDNRR1Sfc52Wfp6pLcpm1NbBZVbe4y5kOXAmsyzTdMOBToJV3IatzV7aEQERZ72YxhZaqIiKEhoYwalQXvvhiIy+/3JOqVUv7OzRjCg1vDrcmAu2AG93+IzglhdxUBbZ79Me7wzKISFXgamByTgsSkTtEZJmILNu/122fMKK8PSu7CDt4MInY2Nk888zCjGH9+zfl44+vsyRhTD7zpuqpjapeJCIrAFT1oIh4U+GbVdsamqn/ZWCEqqbl9JQwVZ0CTAFo2byRswyrdiqSVJX331/Nvfd+w549x4iKCmfo0NaUKRNhT5ozxke8SRQp7nkEhYznUaR7MV88UN2jvxrwd6ZpWgLT3T94BaCPiKSq6oxsl5qa7LyXj/EiBFOYbNy4n8GD5/Ddd38C0LFjDSZNuowyZSL8HJkxhZs3ieIV4HOgkog8DVwLPOzFfEuBeiJSG9gB3ADc5DmBqtY+2S0i04DZOSYJgLQTznuZ2jlOZgqP1NR0nnpqAc8++xMnTqQRHR3JCy/0YMCA5laKMKYAeNPM+HsishzohlOddJWqrvdivlQRGYpzNVMo8KaqrhWRWHd8juclspWe6ryXqHxWs5vgExoqLFy4jRMn0vj3v5szenQPKlSwm+aMKSiimvm0QaYJnKuczqCq23wSUS5a1ovWZbEHoOdb0GSAP0IwBWD37qMkJ6dSs2ZZADZt2s/OnUfp1KmmfwMzJkiJyHJVbXk283pT9TQH5/yEABFAbeB3oPHZrPCcqVuiiIz2y+qNb6WnK1OmLOfBB+fRsmUVvv22PyJCvXrR1Ktn37kx/uBN1dM/PPtF5CJgoM8iyk16mvNe3B50X9jExe0iNnY2ixc7z8cKDw/l6NETREUV93NkxhRteb4zW1V/ExEvb47zAXUvuAq3a+ULiyNHjvPYYz8ybtxi0tOVKlWiGDeuF9dc09BOVhsTALy5M/tej94Q4CJgr88iyo26JQpLFIXCiRNpXHTRFDZvPkBIiHDXXW144olLKF3aShHGBApvShRRHt2pOOcsPvVNOF44WfUUHpXzdCYohIeH0r9/U774YiOTJ19GixZV/B2SMSaTHK96cm+0e05V7y+4kHLWsnqILrtb4a5kCLOjzmCTkpLG2LG/UqNGGW64oQnglCpCQ532mowxvuGTq55EJMy9F+Kisw/NFxRCwy1JBKFFi7YRGzuHNWv2ULFiCfr2rU+pUuH2nAhjAlxOVU9LcM5HxInILOBj4NjJkar6mY9jy16EPcYymBw4kMSIEd8ydeoKAOrUKcfEiX0oVcqeEWFMMPDmHEV5YD/OM7JP3k+hgP8SRYjtYIKBqvLOO6sYPvwb9u1LpFixEEaM6MDIkR2JjCzm7/CMMV7KKVFUcq94WsOpBHFSzrdz+1qoJYpgkJKSzrPP/sS+fYl07lyTSZMuo2FDa/XXmGCTU6IIBUrhXXPhBSvEjkYDVVJSCidOpFGmTATh4aFMmdKXLVsOcsstzeyeCGOCVE6JYqeqPlFgkeRFqCWKQDR37mYGD/6SLl1q8sYbVwLQsWNNOna09pmMCWY5JYrAPfwLi/R3BMbDzp1HuOeeuXz44VoASpYsRmJiCiVKWEI3pjDI6cL1bgUWRV6F2oNqAkFaWjrjxy8hJmYCH364lsjIMEaP7s7y5XdYkjCmEMm2RKGqBwoykDyxEoXfJSen0qnTWyxd6jy0sG/f+rz6am9q1Srr38CMMfkuz40CBoQwK1H4W0REGE2aVGLnzqO88kovrroqxk5WG1NIBWeisKueCpyq8tln66lcuRQXX+w8y2rMmJ6Ehoo1A25MIRekiSI4ww5Wf/55kKFDv+LLLzcRE1OBuLiBFC8eRtmyVrIzpigIzj2uJYoCceJEGi+99DNPPrmApKRUypQpzl13tSEszBrvM6YoCc49rlU9+dzChX8RGzuHdeucR4/cdNM/eOmlSznvvFJ+jswYU9CCNFEEZ9jBIikphWuv/Zg9e45Rt255Jk7sQ48eF/g7LGOMnwTnHleCM+xApqqkpSlhYSFERhZjzJhL2bhxP//9b0ciImx7G1OUBecewJrwyFfr1u0lNnY2PXrU4ZFHOgNw881N/RyVMSZQBOdZSStR5IvExBRGjvyOZs0ms3DhNqZOXcHx46n+DssYE2CCc49rJYpz9tVXmxgy5Ev+/PMQAAMHtuDZZ7tRvHhw/iSMMb4TnHsFO5l91o4dO8GAATP55JN1ADRtWpnJky+jXbvqfo7MGBOognOPa1VPZ61EiWIcOJBEyZLFePzxLtx1V1u7L8IYk6Pg3ONa1VOeLFv2N2XLRlC3bnlEhKlTLyc0NIQaNcr4OzRjTBAIzkNJK1F4JSEhmWHDvqR169eJjZ2NqvNgwtq1y1mSMMZ4LTj3uFaiyJGq8tFHa7n77rns2nWU0FDhoovOJzU1nWLFQv0dnjEmyARnorASRbb++OMAQ4Z8ydy5fwDQrl01Jk/uS9Omlf0cmTEmWAXnHteuesrSkSPHadnydQ4dSqZs2QhGj+7O7bdfREiIPSfCGHP2fLrHFZFewDggFJiqqs9lGn8zMMLtPQoMUtWVuS7YGgXMUlRUce65py2bNx/gxRcvpVKlkv4OyRhTCPgsUYhIKDAB6AHEA0tFZJaqrvOY7E+gs6oeFJHewBSgTa4LtxIFAHv3HuP++7+lW7fa9O/fDIBHHulkT5ozxuQrX1711BrYrKpbVPUEMB240nMCVf1ZVQ+6vb8C1bxachFPFOnpytSpv9GgwXjefnslDz30PSkpaQCWJIwx+c6XiaIqsN2jP94dlp3bgK+yGiEid4jIMhFZBhTpqqc1a/bQqdNb/Oc/X3DwYDLdu9fhu+9usauZjDE+48tD86wObTXLCUUuwUkUF2c1XlWn4FRL0bK6aFEsUSQlpTBq1I+MGfMrqanpVK5ckrFje3LDDU2sFGGM8Slf7nHjAc8GhKoBf2eeSESaAlOB3qq636slF8FEERIizJq1kbS0dAYPbsnTT3ezZ1YbYwqEL/e4S4F6IlIb2AHcANzkOYGI1AA+A/qr6kavlyzBeUN5XsXHH6ZEiWKULx9J8eJhTJvmnOJp08a7UznGGJMffLbHVdVUYCgwF1gPfKSqa0UkVkRi3ckeBaKBiSISl3EOoohLTU1n7NhfaNhwAvff/03G8DZtqlmSMMYUOJ/W4ajql8CXmYZN9ui+Hbg970suvHXyixfHM3DgbFau3A1AQsJxUlPTrYVXY4zfBGdlfyE8eXvoUDIjR37H5MnLUIWaNcswfnwf+vat7+/QjDFFXHAmikLm4MEkGjWayK5dRwkLC2H48HY88kgnSpYM93doxhgTrImicJUoypWLpHfvumzcuJ9Jky7jH/+wBvyMMYEjOBNFkFc9HT+eyujRi+jcuSadO9cCYPz4PkREhFkDfsaYgBOciSKISxTff/8ngwbNYePG/TRsWIHVqwcRGhpCiRJF925zY0xgC9JEEXz27DnG8OHf8O67qwCIianAxImXERpqVzMZYwJbcCaKIKp6OtmA34gR8zh0KJmIiDAefrgj99/fgfBwa5/JGBP4gjNRBFHVU0JCMg899D2HDiXTs+cFTJjQhwsuKO/vsIwxxmtBmigC27FjJwgLC6F48TDKlYtk8uTLSEtTrruukTXgZ4wJOkFaQR64O9tZs36nUaOJPP/8ooxh11zTiOuvb2xJwhgTlIIzUQTgDnfbtgSuumo6V145nW3bEpg79w/S07NsVd0YY4JKcCaKAJKSksaLL/5Mw4YTmDnzd6Kiwhk3rhfz5w+weyKMMYVCkJ6jCIwd8L59iXTr9j9WrXIa8LvuukaMHduTqlVL+zkyY4zJP8GZKAKk6ik6OpIKFUpQu3ZZxo/vQ58+9fwdkgkgKSkpxMfHk5yc7O9QTBESERFBtWrVKFYs/27iDc5E4SeqynvvraZ166rUrx+NiPDuu1dTpkyE3VltzhAfH09UVBS1atWyCxlMgVBV9u/fT3x8PLVr18635QbpOYqC/9P9/vs+und/h/79P2fw4DmoOieqzz8/ypKEyVJycjLR0dGWJEyBERGio6PzvRQbnCWKAvzjJSen8uyzC3nuuUWcOJFGdHQk//xn0wJbvwluliRMQfPFby44E0UBmTdvC4MGzWHz5gMA/PvfzXn++R5ER5fwc2TGGFNwrOopG7t3H6Vv3/fZvPkAjRpVZMGCAbzxxpWWJExQCQ0NpXnz5jRp0oTLL7+cQ4cOZYxbu3YtXbt2pX79+tSrV48nn3wyo0oV4KuvvqJly5Y0bNiQmJgY7rvvPj98gpytWLGC228/i6cpF5Djx4/Tr18/6tatS5s2bdi6dWuW03344Yc0bdqUxo0b88ADD5w27qOPPqJRo0Y0btyYm266CYC9e/fSq1cvX4d/iqoG1atFNVS3z1dfSEtL1/T09Iz+0aN/0mefXajHj6f6ZH2mcFu3bp2/Q9CSJUtmdN9yyy361FNPqapqYmKi1qlTR+fOnauqqseOHdNevXrp+PHjVVV19erVWqdOHV2/fr2qqqakpOiECRPyNbaUlJRzXsa1116rcXFxBbrOvJgwYYIOHDhQVVU/+OADvf7668+YZt++fVq9enXds2ePqjrf07x581RVdePGjdq8eXM9cOCAqqru3r07Y74BAwboTz/9lOV6s/rtAcv0LPe7QVr1lP8liri4XcTGzmbIkFb0798MgAce6JDv6zFF1Es+KgUP9/7u/3bt2rFqldPM/fvvv0+HDh249NJLAShRogTjx4+nS5cuDBkyhOeff56HHnqImJgYAMLCwhg8ePAZyzx69CjDhg1j2bJliAiPPfYY11xzDaVKleLo0aMAfPLJJ8yePZtp06YxYMAAypcvz4oVK2jevDmff/45cXFxlC1bFoC6deuyaNEiQkJCiI2NZdu2bQC8/PLLdOhw+v/xyJEjrFq1imbNnP/rkiVLuPvuu0lKSiIyMpK33nqLBg0aMG3aNObMmUNycjLHjh3jiy++YNiwYaxevZrU1FRGjRrFlVdeydatW+nfvz/Hjh0DYPz48bRv397r7ZuVmTNnMmrUKACuvfZahg4diqqedh5hy5Yt1K9fn4oVKwLQvXt3Pv30U7p168brr7/OkCFDKFeuHACVKlXKmO+qq67ivffeO2O7+EKQJor8c+TIcR577EfGjVtMerpy/Hga//xnUzsJaQqVtLQ0vvvuO2677TbAqXZq0aLFadNccMEFHD16lMOHD7NmzRqGDx+e63KffPJJypQpw+rVqwE4ePBgrvNs3LiRefPmERoaSnp6Op9//jm33norixcvplatWlSuXJmbbrqJe+65h4svvpht27bRs2dP1q9ff9pyli1bRpMmTTL6Y2JiWLBgAWFhYcybN4+RI0fy6aefAvDLL7+watUqypcvz8iRI+natStvvvkmhw4donXr1nTv3p1KlSrx7bffEhERwaZNm7jxxhtZtmzZGfF37NiRI0eOnDH8xRdfpHv37qcN27FjB9WrVwecZFumTBn2799PhQoVMqapW7cuGzZsYOvWrVSrVo0ZM2Zw4sSJjG0F0KFDB9LS0hg1alRGlVPLli15+OGHc93e+SFIE8W578RVlRkzNnDnnV8TH3+YkBDhrrva8MQTl1iSMPkvD0f++SkpKYnmzZuzdetWWrRoQY8ePQDOOKr1lJff/7x585g+fXpG/8kj35xcd911hIY6z2Lp168fTzzxBLfeeivTp0+nX79+Gctdt25dxjyHDx/myJEjREVFZQzbuXNnxlE4QEJCAv/617/YtGkTIkJKSkrGuB49elC+vNO8/zfffMOsWbN48cUXAecy5m3btlGlShWGDh1KXFwcoaGhGTvpzBYuXJjrZzxJ9czvPfP2LVeuHJMmTaJfv36EhITQvn17tmzZAkBqaiqbNm3ixx9/JD4+no4dO7JmzRrKli1LpUqV+Pvvv72O5VwEZ6I4xx35vn2J3HrrTGbPdn4ILVtW4bXX+nLRRefnR3TGBIzIyEji4uJISEigb9++TJgwgTvvvJPGjRuzYMGC06bdsmULpUqVIioqisaNG7N8+fKMap3sZJdwPIdlvqa/ZMmSGd3t2rVj8+bN7N27lxkzZmQcIaenp/PLL78QGRmZ42fzXPYjjzzCJZdcwueff87WrVvp0qVLlutUVT799FMaNGhw2vJGjRpF5cqVWblyJenp6URERGS53ryUKKpVq8b27dupVq0aqampJCQkZCQsT5dffjmXX345AFOmTMlIpNWqVaNt27YUK1aM2rVr06BBAzZt2kSrVq1ITk7OcfvkpyC96uncREWFs3nzAUqXLs748b359dfbLEmYQq1MmTK88sorvPjii6SkpHDzzTfz008/MW/ePMApedx5550ZV9zcf//9PPPMMxlH1enp6YwZM+aM5V566aWMHz8+o/9k1VPlypVZv359RtVSdkSEq6++mnvvvZeGDRsSHR2d5XLj4uLOmLdhw4Zs3rw5oz8hIYGqVasCMG3atGzX2bNnT1599dWMo/0VK1ZkzH/++ecTEhLCO++8Q1paWpbzL1y4kLi4uDNemZMEwBVXXMHbb78NOOdqunbtmmVi3bNnD+Bsv4kTJ2ZcyXXVVVfxww8/ALBv3z42btxInTp1AKdayrPqzZeCNFHkvUSxaNE29u9PBKB48TCmT7+GDRuGMGRIa3tutSkSLrzwQpo1a8b06dOJjIxk5syZPPXUUzRo0IB//OMftGrViqFDhwLQtGlTXn75ZW688UYaNmxIkyZN2Llz5xnLfPjhhzl48CBNmjShWbNmGTu15557jr59+9K1a1fOPz/ng7B+/frx7rvvZlQ7AbzyyissW7aMpk2b0qhRIyZPnnzGfDExMSQkJGQc3T/wwAP897//zajPz84jjzxCSkoKTZs2pUmTJjzyyCMADB48mLfffpu2bduycePG00ohZ+u2225j//791K1blzFjxvDcc89ljGvevHlG91133UWjRo3o0KEDDz74IPXr1wecpBYdHU2jRo245JJLeOGFFzKS6Q8//MBll112zjF6Q7KqQwtkLauLLlv8M1Rp59X0+/cn8uCD85g6dQW33XYhU6de4eMIjXGsX7+ehg0b+juMQm3s2LFERUUF9L0UvtKpUydmzpyZ5XmhrH57IrJcVVuezboK7aG0qvL223HExExg6tQVFCsWQpUqUVmeXDLGBKdBgwZRvHhxf4dR4Pbu3cu9997r1cUD+SE4T2bnUvW0YcM+YmNnM3/+XwB06VKLSZMuIyamQo7zGWOCS0REBP379/d3GAWuYsWKXHXVVQW2vuBMFDlc9RQff5hmzSZz4kQaFSqU4KWXLqV/f7svwvhHTpehGuMLvqg1Cc5EkYNq1UrTv39TQkKE557rTvnyBXP5mDGZRUREsH//fmtq3BQYdZ9Hkd2lvWcrSBPFqT/dzp1HuOeeucTGtqRLl1oATJlyuT2v2vhdtWrViI+PZ+/evf4OxRQhJ59wl5+CM1GIkJaWzqRJy3jooe85fPg4mzcfYOnS/yAiliRMQDh5k5Qxwc6nVz2JSC8R+V1ENovIg1mMFxF5xR2/SkQu8ma5v61KoG3bNxg27CsOHz7O5ZfX59NPr7fivTHG+IDPShQiEgpMAHoA8cBSEZmlqus8JusN1HNfbYBJ7nu2th8qTavei0hPd85HvPpqb668soElCWOM8RFflihaA5tVdYuqngCmA1dmmuZK4H9uc+m/AmVFJMfbOA8kRiIi3HtvW9avH8JVV8VYkjDGGB/y5TmKqsB2j/54ziwtZDVNVeC0tgJE5A7gDrf3ODy2ZswYyKLpmaKmArDP30EECNsWp9i2OMW2xSkNcp8ka75MFFkd5me+wNebaVDVKcAUABFZdra3oRc2ti1OsW1xim2LU2xbnCIiZz5cw0u+rHqKB6p79FcDMjee7s00xhhj/MiXiWIpUE9EaotIOHADMCvTNLOAW9yrn9oCCap6ZhOVxhhj/MZnVU+qmioiQ4G5QCjwpqquFZFYd/xk4EugD7AZSARu9WLRU3wUcjCybXGKbYtTbFucYtvilLPeFkHXzLgxxpiCVWibGTfGGJM/LFEYY4zJUcAmCl81/xGMvNgWN7vbYJWI/CwizfwRZ0HIbVt4TNdKRNJE5NqCjK8gebMtRKSLiMSJyFoRmV/QMRYUL/4jZUTkCxFZ6W4Lb86HBh0ReVNE9ojImmzGn91+U1UD7oVz8vsPoA4QDqwEGmWapg/wFc69GG2Bxf6O24/boj1Qzu3uXZS3hcd03+NcLHGtv+P24++iLLAOqOH2V/J33H7cFiOB0W53ReAAEO7v2H2wLToBFwFrshl/VvvNQC1R+KT5jyCV67ZQ1Z9V9aDb+yvO/SiFkTe/C4BhwKfAnoIMroB5sy1uAj5T1W0AqlpYt4c320KBKHHa+ymFkyhSCzZM31PVBTifLTtntd8M1ESRXdMeeZ2mMMjr57wN54ihMMp1W4hIVeBqYHIBxuUP3vwu6gPlRORHEVkuIrcUWHQFy5ttMR5oiHND72rgLlVNL5jwAspZ7TcD9XkU+db8RyHg9ecUkUtwEsXFPo3If7zZFi8DI1Q1rZA3FunNtggDWgDdgEjgFxH5VVU3+jq4AubNtugJxAFdgQuAb0Vkoaoe9nFsgeas9puBmiis+Y9TvPqcItIUmAr0VtX9BRRbQfNmW7QEprtJogLQR0RSVXVGgURYcLz9j+xT1WPAMRFZADQDClui8GZb3Ao8p05F/WYR+ROIAZYUTIgB46z2m4Fa9WTNf5yS67YQkRrAZ0D/Qni06CnXbaGqtVW1lqrWAj4BBhfCJAHe/UdmAh1FJExESuC03ry+gOMsCN5si204JStEpDJOS6pbCjTKwHBW+82ALFGo75r/CDpebotHgWhgonsknaqFsMVML7dFkeDNtlDV9SLyNbAKSAemqmqWl00GMy9/F08C00RkNU71ywhVLXTNj4vIB0AXoIKIxAOPAcXg3Pab1oSHMcaYHAVq1ZMxxpgAYYnCGGNMjixRGGOMyZElCmOMMTmyRGGMMSZHlihMQHJbfo3zeNXKYdqj+bC+aSLyp7uu30Sk3VksY6qINHK7R2Ya9/O5xugu5+R2WeO2hlo2l+mbi0if/Fi3Kbrs8lgTkETkqKqWyu9pc1jGNGC2qn4iIpcCL6pq03NY3jnHlNtyReRtYKOqPp3D9AOAlqo6NL9jMUWHlShMUBCRUiLynXu0v1pEzmg1VkTOF5EFHkfcHd3hl4rIL+68H4tIbjvwBUBdd9573WWtEZG73WElRWSO+2yDNSLSzx3+o4i0FJHngEg3jvfccUfd9w89j/Ddksw1IhIqIi+IyFJxnhMw0IvN8gtug24i0lqcZ5GscN8buHcpPwH0c2Pp58b+prueFVltR2PO4O/20+1lr6xeQBpOI25xwOc4rQiUdsdVwLmz9GSJ+Kj7Phx4yO0OBaLcaRcAJd3hI4BHs1jfNNxnVwDXAYtxGtRbDZTEaZp6LXAhcA3wuse8Zdz3H3GO3jNi8pjmZIxXA2+73eE4LXlGAncAD7vDiwPLgNpZxHnU4/N9DPRy+0sDYW53d+BTt3sAMN5j/meAf7rdZXHafSrp7+/bXoH9CsgmPIwBklS1+ckeESkGPCMinXCao6gKVAZ2ecyzFHjTnXaGqsaJSGegEbDIbd4kHOdIPCsviMjDwF6cVni7AZ+r06geIvIZ0BH4GnhRREbjVFctzMPn+gp4RUSKA72ABaqa5FZ3NZVTT+QrA9QD/sw0f6SIxAG1gOXAtx7Tvy0i9XBaAy2WzfovBa4Qkfvc/gigBoWzDSiTTyxRmGBxM86TyVqoaoqIbMXZyWVQ1QVuIrkMeEdEXgAOAt+q6o1erON+Vf3kZI+IdM9qIlXdKCItcNrMeVZEvlHVJ7z5EKqaLCI/4jR73Q/44OTqgGGqOjeXRSSpanMRKQPMBoYAr+C0ZfSDql7tnvj/MZv5BbhGVX/3Jl5jwM5RmOBRBtjjJolLgJqZJxCRmu40rwNv4DwS8legg4icPOdQQkTqe7nOBcBV7jwlcaqNFopIFSBRVd8FXnTXk1mKW7LJynScxtg64jRkh/s+6OQ8IlLfXWeWVDUBuBO4z52nDLDDHT3AY9IjOFVwJ80FholbvBKRC7NbhzEnWaIwweI9oKWILMMpXWzIYpouQJyIrMA5jzBOVffi7Dg/EJFVOIkjxpsVqupvOOculuCcs5iqqiuAfwBL3Cqgh4Cnsph9CrDq5MnsTL7BebbxPHUe3QnOs0TWAb+JyBrgNXIp8buxrMRpVvt5nNLNIpzzFyf9ADQ6eTIbp+RRzI1tjdtvTI7s8lhjjDE5shKFMcaYHFmiMMYYkyNLFMYYY3JkicIYY0yOLFEYY4zJkSUKY4wxObJEYYwxJkf/D/MYAKWWjFT1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_score=clf_class.predict_proba(train)\n",
    "y_test=np.zeros((200000,2))\n",
    "for i in range(200000):\n",
    "    if y[i,0]==0:\n",
    "        y_test[i,0]=1\n",
    "    else:\n",
    "        y_test[i,1]=1\n",
    "\n",
    "from sklearn.metrics import roc_curve, auc\n",
    "fpr = dict()\n",
    "tpr = dict()\n",
    "roc_auc = dict()\n",
    "for i in range(2):   #两类\n",
    "    #计算每个target的fpr和tpr\n",
    "    fpr[i], tpr[i], _ = roc_curve(y_test[:, i], y_score[:, i])  #y_test ont-hot\n",
    "    roc_auc[i] = auc(fpr[i], tpr[i])  # auc直接计算面积\n",
    "    \n",
    "# Compute micro-average ROC curve and ROC area\n",
    "fpr[\"micro\"], tpr[\"micro\"], _ = roc_curve(y_test.ravel(), y_score.ravel())\n",
    "roc_auc[\"micro\"] = auc(fpr[\"micro\"], tpr[\"micro\"])\n",
    "\n",
    "plt.figure()\n",
    "id=1  # 选择不同的id即绘制不同target分类的效果\n",
    "lw = 2\n",
    "plt.plot(fpr[id], tpr[id], color='darkorange',\n",
    "         lw=lw, label='ROC curve (area = %0.2f)' % roc_auc[1])#S1=0.96\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver operating characteristic example')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 随机森林模型分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;为了得到最适合解决此问题的分类模型，本人查找相关网站，发现随机森林算法是解决该问题的最佳模型，因此，本人设计随机森林算法对交通事故理赔预测问题进行解决。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 随机森林算法原理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;随机森林是一种很灵活实用的方法，它有如下几个特点：<br>\n",
    "* 在当前所有算法中，具有极好的准确率<br>\n",
    "* 能够有效地运行在大数据集上\n",
    "* 能够处理具有高维特征的输入样本，而且不需要降维\n",
    "* 能够评估各个特征在分类问题上的重要性\n",
    "* 在生成过程中，能够获取到内部生成误差的一种无偏估计\n",
    "* ......"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 随机森林的相关基础知识"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1）、信息、熵以及信息增益的概念<br>\n",
    "&ensp;&ensp;&ensp;&ensp;对于机器学习中的决策树而言，如果带分类的事物集合可以划分为多个类别当中，则某个类（xi）的信息可以定义如下:<br>\n",
    "$$I(x=x_I)=\\log_2P(x_i)$$\n",
    "&ensp;&ensp;&ensp;&ensp;I(x)用来表示随机变量的信息，p(xi)指是当xi发生时的概率。<br>\n",
    "&ensp;&ensp;&ensp;&ensp;熵是用来度量不确定性的，当熵越大，X=xi的不确定性越大，反之越小。对于机器学习中的分类问题而言，熵越大即这个类别的不确定性更大，反之越小。<br>\n",
    "&ensp;&ensp;&ensp;&ensp;信息增益在决策树算法中是用来选择特征的指标，信息增益越大，则这个特征的选择性越好。<br>\n",
    "#### 2) 、决策树<br>\n",
    "&ensp;&ensp;&ensp;&ensp;决策树是一种树形结构，其中每个内部节点表示一个属性上的测试，每个分支代表一个测试输出，每个叶节点代表一种类别。常见的决策树算法有C4.5、ID3和CART。\n",
    "#### 3) 、 随机森林的生成<br>\n",
    "&ensp;&ensp;&ensp;&ensp;前面提到，随机森林中有许多的分类树。我们要将一个输入样本进行分类，我们需要将输入样本输入到每棵树中进行分类。打个形象的比喻：森林中召开会议，讨论某个动物到底是老鼠还是松鼠，每棵树都要独立地发表自己对这个问题的看法，也就是每棵树都要投票。该动物到底是老鼠还是松鼠，要依据投票情况来确定，获得票数最多的类别就是森林的分类结果。森林中的每棵树都是独立的，99.9%不相关的树做出的预测结果涵盖所有的情况，这些预测结果将会彼此抵消。少数优秀的树的预测结果将会超脱于芸芸“噪音”，做出一个好的预测。将若干个弱分类器的分类结果进行投票选择，从而组成一个强分类器，这就是随机森林bagging的思想（关于bagging的一个有必要提及的问题：bagging的代价是不用单棵决策树来做预测，具体哪个变量起到重要作用变得未知，所以bagging改进了预测准确率但损失了解释性。）。下图可以形象地描述这个情况：\n",
    "![](https://images0.cnblogs.com/blog2015/764050/201506/182310220134010.png)\n",
    "&ensp;&ensp;&ensp;&ensp;有了树我们就可以分类了，但是森林中的每棵树是怎么生成的呢?<br>\n",
    "&ensp;&ensp;&ensp;&ensp;每棵树的按照如下规则生成:<br>\n",
    "&ensp;&ensp;&ensp;&ensp;1）如果训练集大小为N，对于每棵树而言，随机且有放回地从训练集中的抽取N个训练样本（这种采样方式称为bootstrap sample方法），作为该树的训练集；<br>\n",
    "&ensp;&ensp;&ensp;&ensp;从这里我们可以知道：每棵树的训练集都是不同的，而且里面包含重复的训练样本。<br>\n",
    "&ensp;&ensp;&ensp;&ensp;如果不进行随机抽样，每棵树的训练集都一样，那么最终训练出的树分类结果也是完全一样的，这样的话完全没有bagging的必要；<br>\n",
    "&ensp;&ensp;&ensp;&ensp;如果不是有放回的抽样，那么每棵树的训练样本都是不同的，都是没有交集的，这样每棵树都是\"有偏的\"，都是绝对\"片面的\"（当然这样说可能不对），也就是说每棵树训练出来都是有很大的差异的；而随机森林最后分类取决于多棵树（弱分类器）的投票表决，这种表决应该是\"求同\"，因此使用完全不同的训练集来训练每棵树这样对最终分类结果是没有帮助的，这样无异于是\"盲人摸象\"。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;2）如果每个样本的特征维度为M，指定一个常数m<<M，随机地从M个特征中选取m个特征子集，每次树进行分裂时，从这m个特征中选择最优的；\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;3）每棵树都尽最大程度的生长，并且没有剪枝过程。\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;因此，我们提到的随机森林中的“随机”就是指的这里的两个随机性。两个随机性的引入对随机森林的分类性能至关重要。由于它们的引入，使得随机森林不容易陷入过拟合，并且具有很好得抗噪能力（比如：对缺省值不敏感）。\n",
    "#### 4）、随机森林分类效果（错误率）与两个因素有关：\n",
    "* 森林中任意两棵树的相关性：相关性越大，错误率越大；\n",
    "* 森林中每棵树的分类能力：每棵树的分类能力越强，整个森林的错误率越低。\n",
    "&ensp;&ensp;&ensp;&ensp;减小特征选择个数m，树的相关性和分类能力也会相应的降低；增大m，两者也会随之增大。所以关键问题是如何选择最优的m（或者是范围），这也是随机森林唯一的一个参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 随机森林模型的代码实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-6-39f652175813>:18: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().\n",
      "  rfc = rfc.fit(train,y)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.93115\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier \n",
    "import numpy as np\n",
    "num,q1,q2,q3,q4,q5,q6,q7,q8,q9,q10,q11,q12,q13,q14,q15,q16,q17,q18,q19,q20,q21,q22,q23,q24,q25,q26,q27,q28,q29,q30,q31,q32,q33,q34,q35,q36,y= np.loadtxt(\"train.csv\",delimiter=',', usecols=(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37), \n",
    "\tunpack=True)\n",
    "num = num.reshape(-1,1)\n",
    "y = y.reshape(-1,1)\n",
    "for i in range(1,37):\n",
    "    exec (\"q%s=q%s.reshape(-1,1)\"%(i,i))\n",
    "train = np.hstack((q1.reshape(-1,1),q2.reshape(-1,1)))\n",
    "for i in range(3,37):\n",
    "    exec (\"train=np.hstack((train,q%s.reshape(-1,1)))\"%i)\n",
    "#产生训练集\n",
    "\n",
    "\n",
    "# 建立随机森林模型\n",
    "rfc = RandomForestClassifier(n_estimators=10, random_state=0)\n",
    "#第一个参数代表随机森林中树木的个数，往往是越多越好，但准确率会收敛，参数越大，计算时间越长\n",
    "rfc = rfc.fit(train,y)       \n",
    "#用训练集数据训练模型 \n",
    "\n",
    "y_pred=[]\n",
    "j=0\n",
    "for i in range(np.shape(train)[0]):\n",
    "    y_pred.append(rfc.predict([train[i]]))\n",
    "for i in range(np.shape(train)[0]):\n",
    "    if y_pred[i]==y[i]:\n",
    "        j=j+1\n",
    "print(j/200000)\n",
    "#训练集准确率计算：0.931（随机森林中有10棵树木）\n",
    "#训练集准确率计算：0.931（随机森林中有20棵树木）\n",
    "#但是因此而带来巨大的时间代价，但是准确率几乎没有提高，综合来看，10棵树木就能符合要求"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 将预测数据标签写入test.csv文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Caseld Evaluation\n",
      "0   200001       [0.]\n",
      "1   200002       [0.]\n",
      "2   200003       [0.]\n",
      "3   200004       [1.]\n",
      "4   200005       [0.]\n",
      "..     ...        ...\n",
      "95  200096       [0.]\n",
      "96  200097       [1.]\n",
      "97  200098       [0.]\n",
      "98  200099       [0.]\n",
      "99  200100       [0.]\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "f = open('sample_submit_rd.csv','w',encoding='utf-8',newline='' \"\")\n",
    "csv_writer = csv.writer(f)\n",
    "csv_writer.writerow([\"Caseld\",\"Evaluation\"])\n",
    "for i in range(80000):\n",
    "    n=200000+i+1\n",
    "    csv_writer.writerow([n,y_pred[i]])#数据写入\n",
    "import pandas as pd\n",
    "sheet = pd.read_csv('sample_submit_rd.csv',encoding='utf-8')\n",
    "print(sheet.head(100))\n",
    "#打印表格前100行作为举例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 将随机森林的错误数据可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAFzCAYAAAAwtX3sAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAwFUlEQVR4nO3deZyWdb3/8dcHRhEXVgUJNDFpUY+lFmrbMUnRMjHDxEw5aVHmWvYrzcrS0zlqmWalRbngkorkgrukmXVyTXPPpOwospks7sDA5/fHfeG5r2EYxhlmBuZ6PXtcj7nv77V977vHOG8+3+91XZGZSJIkVV2Pru6AJEnSmsBQJEmShKFIkiQJMBRJkiQBhiJJkiTAUCRJkgRAQ1d3YGV6b3+k9wqQusD8+37a1V2QKmu9BqIzz9fev7WvPfjTTu1vR1tjQ5EkSepg4YBRPb8NSZIkDEWSJFVXRPuWVR4+zo+IuRHxaJP2oyLiyYh4LCJOr2s/ISKmF+tG17XvGBGPFOvOjqidPCJ6RcQVRfs9EbFF3T7jI+KpYhnfmq/DUCRJUlVFj/Ytq3YhsGfplBEfAcYA22XmNsAPi/atgXHANsU+50REz2K3c4EJwIhiWX7Mw4D5mbkVcCZwWnGsAcBJwE7ASOCkiOi/qs4aiiRJqqoOrhRl5p3AvCbNhwOnZuaiYpu5RfsY4PLMXJSZTwPTgZERMQTok5l3Ze2BrRcB+9btM6l4PQUYVVSRRgPTMnNeZs4HptEknDXHUCRJktokIiZExP11y4RW7PZ24EPFcNfvI+J9RftQ4Nm67WYUbUOL103bS/tkZiOwEBjYwrFa5NVnkiRVVTuvPsvMicDEN7lbA9Af2Bl4HzA5IraEZm9HkC2008Z9VspKkSRJVdXBw2crMQO4KmvuBZYBGxftm9VtNwyYWbQPa6ad+n0iogHoS224bmXHapGhSJKkqur4idbNuQbYDSAi3g6sC/wLmAqMK64oG05tQvW9mTkLeCkidi7mCx0CXFscayqw/MqyscDtxbyjW4A9IqJ/McF6j6KtRQ6fSZJUVW2v9rTy8HEZsCuwcUTMoHZF2PnA+cVl+ouB8UWQeSwiJgOPA43AEZm5tDjU4dSuZOsN3FQsAOcBF0fEdGoVonEAmTkvIk4B7iu2Ozkzm074XrG/tX6seXzMh9Q1fMyH1HU6/TEfuxzfvsd83HWqj/mQJEndgI/5KDEUSZJUVR08fLa2MRRJklRVVopKDEWSJFWVlaISI6IkSRJWiiRJqi6Hz0oMRZIkVZWhqMRQJElSVfVwTlE9I6IkSRJWiiRJqi6Hz0oMRZIkVZWX5JcYiiRJqiorRSWGIkmSqspKUYkRUZIkCStFkiRVl8NnJYYiSZKqyuGzEkORJElVZaWoxFAkSVJVWSkqMSJKkiRhpUiSpOpy+KzEUCRJUlU5fFZiKJIkqaqsFJX4bUiSJGGlSJKk6rJSVGIokiSpqpxTVGIokiSpqqwUlRiKJEmqKitFJUZESZIkrBRJklRdDp+VGIokSaoqh89KDEWSJFVUGIpKDEWSJFWUoajMwURJktQhIuL8iJgbEY82s+5rEZERsXFd2wkRMT0inoyI0XXtO0bEI8W6s6NIcxHRKyKuKNrviYgt6vYZHxFPFcv41vTXUCRJUlVFO5dVuxDYc4XTRmwG7A48U9e2NTAO2KbY55yI6FmsPheYAIwoluXHPAyYn5lbAWcCpxXHGgCcBOwEjAROioj+q+qsoUiSpIqKiHYtq5KZdwLzmll1JvB1IOvaxgCXZ+aizHwamA6MjIghQJ/MvCszE7gI2Ldun0nF6ynAqKKKNBqYlpnzMnM+MI1mwllTzimSJKmiumJOUUTsAzyXmQ81Of9Q4O669zOKtiXF66bty/d5FiAzGyNiITCwvr2ZfVbKUCRJktokIiZQG9ZabmJmTmxh+/WBE4E9mlvdTFu20N7WfVbKUCRJUkW1t1JUBKCVhqBmvA0YDiyvEg0DHoiIkdSqOZvVbTsMmFm0D2umnbp9ZkREA9CX2nDdDGDXJvvcsarOOadIkqSK6ug5RU1l5iOZOSgzt8jMLaiFlx0yczYwFRhXXFE2nNqE6nszcxbwUkTsXMwXOgS4tjjkVGD5lWVjgduLeUe3AHtERP9igvUeRVuLrBRJklRVHTylKCIuo1ax2TgiZgAnZeZ5zW2bmY9FxGTgcaAROCIzlxarD6d2JVtv4KZiATgPuDgiplOrEI0rjjUvIk4B7iu2Ozkzm5vwXe5vLVCteXpvf+Sa2TGpm5t/30+7ugtSZa3X0NExpazfQZe062/tgks/263u/ujwmSRJEg6fSZJUWT7mo8xQJElSRRmKygxFkiRVlKGozFAkSVJVmYlKnGgtSZKElSJJkirL4bMyQ5EkSRVlKCozFEmSVFGGojLnFEmSJGGlSJKk6rJQVGIokiSpohw+KzMUSZJUUYaiMkORJEkVZSgqc6K1JEkSVookSaosK0VlhiJJkqrKTFRiKJIkqaKsFJUZiiRJqihDUZkTrSVJkrBSJElSZVkpKjMUSZJUVWaiEkORJEkVZaWozDlFkiRJWCkS8POTDmKvD2/L8/Ne4r37/1ez23xoxxH84P99inUaevLCgpfZ4/M/btc5112ngfNOOZjt37U58xa+wme/cT7PzJoHwMv3n82j02cC8Ozs+ex/7C/adS5pTbbX7rux/gYb0LNHD3o29OSyyVeV1t9w/VQuOO+XAKy//gac+O3v8o53vrNd51y8eDEnnvB1nnjsMfr268fpZ5zJ0KHDmDnzOb56zFEsW7qUJY2NHHjQZ/n0AQe261xas1kpKjMUiYuvu5ufX/F7fnXKIc2u77thb378zU8z5ohzeHb2fDbpv2Grj735kAH88uSDGf2Fcoj6j313Yf5Lr7HtmO+x/+gd+f4xYzj4+AsAeG3REnYed2rbP5C0lvnVBZPo339As+uGDh3G+RdeQp++ffnjH37Pyd/9NpdefmWrjvvcczP4zokncN6FF5far/7NlfTp04frb57GTTfewFk/+iE/OOMsNtl4Ey669HLWXXddXn3lFT617yfY9SO7MWjQ4HZ/Rq2ZDEVlHRaKIuKdwBhgKJDATGBqZj7RUedU2/zPA39n8yHN/wcZ4IC93su1tz3Es7PnA/D8/JffWDfuY+/jiAP/nXXWaeC+R/7JMf99BcuW5SrPufeu2/H9X9wIwFW/fZAffWP/dn4KqXt6z/Y7vPF6u+3ew5w5s994f/111/LrSy6mcckStt3u3Zz47ZPo2bPnKo/5u9tv5/AjjgRg9z1Gc+r3TyYzWWfddd/YZvGSxSxbtmw1fhKtiQxFZR0ypygivgFcTm1e+73AfcXryyLi+I44pzrOiLcOol+f9bnll8fwP5d+nc/sPRKAdwwfzNg9duAjn/sRO487laXLljHuY+9r1THfMqgvM4qQtXTpMl58+TUG9tsAgPXWbeCPl36d3086jk/sul3HfChpTRHwpS8cxrj992PK5Cta3PTqq6bwwQ99GIB//P3v3HLTTUy65DImX3UtPXv04Mbrr2vVKefOncOmmw4BoKGhgQ032ogFC2q/j7NnzWLsJz/B6FG78rnDvmCVqLuLdi7dTEdVig4DtsnMJfWNEfEj4DGg2bGRiJgATABoGLYrDRtv00Hd05vR0LMHO7xrM/b64k/ovd463DHpOO59+J98ZOQ72GHrzfnjJV8HoHevdXh+Xq2KdMUZX+CtQwey7jo92WzTAdx9eS0L/+zXd3Dx1Lub/ddJFgWmt3/sO8x6fiFbDB3IzROP5tHpM3l6xr8658NKnWzSJZcxaNBgXnjhBb70+c8xfMst2fG9K/7j4t577ubqq6Zw4cW/BuCeu+/iiccf5aADxgLw+qLXGTBwIADHHn0EM2fMYMmSJcyaNYtP7zcGgM8cfAj7fvJTZK5YzV3+O7npkCFMufo65s6dw7FHHcHue4xm4MYbd8hnl9Y0HRWKlgFvAf63SfuQYl2zMnMiMBGg9/ZHrnoMRp3iubkL+NeCV3j19cW8+vpi/vjAdLZ7+1Aigkuuu4fv/GTqCvsccFxtYujK5hQ9N2cBwzbtz3NzF9CzZw/6bNibeQtfAWDW8wsB+OdzL3Dn/U/xnncOMxSp21peiRk4cCC7fXR3Hn3k4RVC0d+e/CvfO+lb/Oznv6Rfv/4AJMknxnySY75y3ArHPOvsnwErn1M0ePCmzJ49i8GbbkpjYyMvv/QSffv2W6Ffb9tqBA/8+X52H73n6vq4WsM4fFbWUZfkHwvcFhE3RcTEYrkZuA04poPOqQ5y3R0P84Ht30bPnj3ovd46vG/bLfjr07P53b1P8smPvueNidf9+6zP5kP6t+qYN/z+EQ76xE4A7PfR7fn9fX8DoN9GvVl3nVpWH9hvA3Z5z5Y88Y/ZKz2OtDZ79dVXeeWVl994fdef/oetthpR2mbWzJl89Zij+P5/n84WWwx/o32nnXbht7fewgsvvADAwgULmDnzuVadd9eP7MbUa68GYNqttzByp52JCObMns3rr78OwIsLF/KXBx9gi+HDWzqU1nIR0a6lu+mQSlFm3hwRbwdGUptoHcAM4L7MXNoR51TbTfrv/+BDO45g434bMv3mUzjl5zeyTkNtsuavpvyRJ5+ew7Q/Pc59k09g2bLkwqv/xON/nwXA9352PdedeyQ9IljSuJSvnDqZZ2bNX+U5L7zmT5z/n4fw6LUnMf/FV9648uydW27KT048kGW5jB7Rgx9eMI2/GorUTc174QW+cvQRADQuXcrHPr43H/jQh5l8xWUAfPqAA/nFz3/GgoUL+K9TvgfwxmX7b9tqK444+lgO/8KhLMtlNDSswze/9R3e8pahqzzvJz81lhOP/3/svefu9Onbl9N/eCYA//jH3znjB6cSBEky/j8OZcTb39FBn15rgm6Ya9olmhtbXhM4fCZ1jfn3/bSruyBV1noNnTt9eauv3dSuv7XTf7hXi/2NiPOBvYG5mblt0fYD4BPAYuDvwOcyc0Gx7gRq85KXAkdn5i1F+47AhUBv4EbgmMzMiOgFXATsCLwAHJCZ/yz2GQ98q+jKf2bmpFV9Hu9oLUlSRXXC8NmFQNNJadOAbTNzO+BvwAlFX7YGxgHbFPucExHL7zFxLrULsUYUy/JjHgbMz8ytgDOB04pjDQBOAnaiNmp1UkSscn6HoUiSpIqKaN+yKpl5JzCvSdutmdlYvL0bGFa8HgNcnpmLMvNpYDowMiKGAH0y866sDW9dBOxbt8/yCtAUYFTU0tpoYFpmzsvM+dSC2CqvGPCO1pIkVdQaMFn6UGD5DbqGUgtJy80o2pYUr5u2L9/nWYDMbIyIhcDA+vZm9lkpK0WSJFVUeytFETEhIu6vWya0/txxItAIXLq8qZnNsoX2tu6zUlaKJElSm9TfX/DNKCZB7w2Myv+74msGsFndZsOoPSJsBv83xFbfXr/PjIhoAPpSG66bAezaZJ87VtUvK0WSJFVUjx7RrqUtImJP4BvAPpn5at2qqcC4iOgVEcOpTai+NzNnAS9FxM7FfKFDgGvr9hlfvB4L3F6ErFuAPSKifzHBeo+irUVWiiRJqqiOnlIUEZdRq9hsHBEzqF0RdgLQC5hWzGm6OzO/lJmPRcRk4HFqw2pH1N3b8HD+75L8m4oF4Dzg4oiYTq1CNA4gM+dFxCnUnr0KcHJmliZ8N9tf71MkqZ73KZK6Tmffp2jbb01r19/aR/9z9y6fqb06WSmSJKmiuv7iszWLc4okSZKwUiRJUmWtAfcpWqMYiiRJqihDUZmhSJKkijITlTmnSJIkCStFkiRVlsNnZYYiSZIqykxUZiiSJKmirBSVGYokSaooM1GZE60lSZKwUiRJUmU5fFZmKJIkqaLMRGWGIkmSKspKUZmhSJKkijITlTnRWpIkCStFkiRVlsNnZYYiSZIqykxUZiiSJKmirBSVOadIkiQJK0WSJFWWhaIyQ5EkSRXl8FmZoUiSpIoyFJUZiiRJqigzUZkTrSVJkrBSJElSZTl8VmYokiSposxEZYYiSZIqykpRmaFIkqSKMhOVOdFakiQJK0WSJFVWD0tFJYYiSZIqykxUZiiSJKminGhd5pwiSZLUISLi/IiYGxGP1rUNiIhpEfFU8bN/3boTImJ6RDwZEaPr2neMiEeKdWdHkeYioldEXFG03xMRW9TtM744x1MRMb41/TUUSZJUUT2ifUsrXAjs2aTteOC2zBwB3Fa8JyK2BsYB2xT7nBMRPYt9zgUmACOKZfkxDwPmZ+ZWwJnAacWxBgAnATsBI4GT6sPXSr+PVn0kSZLU7UREu5ZVycw7gXlNmscAk4rXk4B969ovz8xFmfk0MB0YGRFDgD6ZeVdmJnBRk32WH2sKMKqoIo0GpmXmvMycD0xjxXC2AkORJEkVFdHeJSZExP11y4RWnHZwZs4CKH4OKtqHAs/WbTejaBtavG7aXtonMxuBhcDAFo7VIidaS5JUUUH7Jlpn5kRg4urpTbOdyRba27rPSlkpkiRJnWlOMSRG8XNu0T4D2Kxuu2HAzKJ9WDPtpX0iogHoS224bmXHapGhSJKkiuqEidbNmQosvxpsPHBtXfu44oqy4dQmVN9bDLG9FBE7F/OFDmmyz/JjjQVuL+Yd3QLsERH9iwnWexRtLXL4TJKkiuro+xRFxGXArsDGETGD2hVhpwKTI+Iw4Blgf4DMfCwiJgOPA43AEZm5tDjU4dSuZOsN3FQsAOcBF0fEdGoVonHFseZFxCnAfcV2J2dm0wnfKzAUSZJUUR1978bMPHAlq0atZPvvA99vpv1+YNtm2l+nCFXNrDsfOL/VncVQJElSZfnss7I3NaeoGJvbrqM6I0mS1FVWGYoi4o6I6FPcHfIh4IKI+FHHd02SJHWk9t6nqLtpTaWob2a+COwHXJCZOwIf7dhuSZKkjtbRd7Re27QmFDUU9xH4NHB9B/dHkiR1EitFZa0JRSdTu7Z/embeFxFbAk91bLckSZI61yqvPsvMK4Er697/A/hUR3ZKkiR1PK8+K1tpKIqIn9DCc0Iy8+gO6ZEkSeoURqKylipF93daLyRJUqfrjpOl22OloSgzJ9W/j4gNMvOVju+SJEnqDO14flm31Jr7FO0SEY8DTxTv3x0R53R4zyRJkjpRa64+OwsYDbwAkJkPAR/uwD5JkqRO4H2Kylr17LPMfLbJh1+6sm0lSdLaoRvmmnZpTSh6NiLeD2RErAscTTGUJkmS1l7dsdrTHq0JRV8CfgwMBZ6jdiPHIzqyU5IkqeM50bqsNTdv/BdwUCf0RZIkqcu05uqzLSPiuoh4PiLmRsS1xaM+JEnSWsyJ1mWtufrs18BkYAjwFmqP/LisIzslSZI6XrRz6W5aE4oiMy/OzMZiuYQWHv8hSZLWDj0i2rV0Ny09+2xA8fJ3EXE8cDm1MHQAcEMn9E2SJKnTtDTR+s/UQtDyKPjFunUJnNJRnZIkSR2vGxZ72qWlZ58N78yOSJKkztUdJ0u3R6vuaB0R2wJbA+stb8vMizqqU5IkqeOZicpWGYoi4iRgV2qh6EZgL+CPgKFIkqS1WHecLN0erbn6bCwwCpidmZ8D3g306tBeSZIkdbLWDJ+9lpnLIqIxIvoAcwFv3ihJ0lrOQlFZa0LR/RHRD/gltSvSXgbu7chOATx/9086+hSSmjH3xUVd3QWpsjYf0LkDMU60LmvNs8++XLz8eUTcDPTJzIc7tluSJKmjtWYOTZW0dPPGHVpal5kPdEyXJElSZ7BSVNZSpeiMFtYlsNtq7oskSVKXaenmjR/pzI5IkqTO1cNCUUmrbt4oSZK6H0NRmXOsJEmqqIho19KK438lIh6LiEcj4rKIWC8iBkTEtIh4qvjZv277EyJiekQ8GRGj69p3jIhHinVnR3HyiOgVEVcU7fdExBbt+T4MRZIkabWLiKHA0cB7M3NboCcwDjgeuC0zRwC3Fe+JiK2L9dsAewLnRETP4nDnAhOAEcWyZ9F+GDA/M7cCzgROa0+fVxmKouazEfGd4v3mETGyPSeVJEldr0e0b2mFBqB3RDQA6wMzgTHApGL9JGDf4vUY4PLMXJSZTwPTgZERMYTa7YDuysyk9pix+n2WH2sKMGp5FalN30crtjkH2AU4sHj/EvCztp5QkiStGSLau8SEiLi/bpmw/NiZ+RzwQ+AZYBawMDNvBQZn5qxim1nAoGKXocCzdd2bUbQNLV43bS/tk5mNwEJgYFu/j9ZMtN4pM3eIiAeLk86PiHXbekJJkrRmaO8DYTNzIjCxuXXFXKExwHBgAXBlRHy2hcM115lsob2lfdqkNZWiJcWYXgJExCbAsraeUJIkrRl6tHNZhY8CT2fm85m5BLgKeD8wpxgSo/g5t9h+BrBZ3f7DqA23zSheN20v7VMM0fUF5rXu06+oNaHobOBqYFBEfB/4I/BfbT2hJEmqhGeAnSNi/WKezyjgCWAqML7YZjxwbfF6KjCuuKJsOLUJ1fcWQ2wvRcTOxXEOabLP8mONBW4v5h21SWuefXZpRPy5+DAB7JuZT7T1hJIkac3QkU/5yMx7ImIK8ADQCDxIbahtQ2ByRBxGLTjtX2z/WERMBh4vtj8iM5cWhzscuBDoDdxULADnARdHxHRqFaJx7elzrCpQRcTmzbVn5jPtOfGqvLyo7UlPUtvNe2VxV3dBqqzNB/Tq1Nspfvvmp9r1t/aUPUd0q9s/tmai9Q3830Sn9ahNmHqS2n0EJEnSWsrnwZa1Zvjs3+rfR8QOwBc7rEeSJEld4E0/+ywzH4iI93VEZyRJUufx2WdlqwxFEfHVurc9gB2A5zusR5IkqVO09z5F3U1rKkUb1b1upDbH6Dcd0x1JktRZzERlLYai4qaNG2bm/+uk/kiSpE7i8FnZSm/eGBENxf0BdujE/kiSJHWJlipF91ILRH+JiKnAlcAry1dm5lUd3DdJktSBotlHh1VXa+YUDQBeAHbj/+5XlNSeYSJJktZSDp+VtRSKBhVXnj3Kik+p9W7TkiSt5QxFZS2Fop7Unk/S3FdmKJIkaS0XXn5W0lIompWZJ3daTyRJkrpQS6HI+ChJUjfm8FlZS6FoVKf1QpIkdTpHz8pWGooyc15ndkSSJHUuH/NRttKbN0qSJFVJa+5TJEmSuiHnFJUZiiRJqihHz8oMRZIkVVQPLzQvMRRJklRRVorKnGgtSZKElSJJkirLidZlhiJJkirK+xSVGYokSaooM1GZoUiSpIqyUlTmRGtJkiSsFEmSVFkWisoMRZIkVZTDRWWGIkmSKiosFZUYEiVJkrBSJElSZVknKjMUSZJUUV6SX+bwmSRJFRXtXFp1joh+ETElIv4aEU9ExC4RMSAipkXEU8XP/nXbnxAR0yPiyYgYXde+Y0Q8Uqw7O4oJURHRKyKuKNrviYgt2vp9GIokSaqoiPYtrfRj4ObMfCfwbuAJ4HjgtswcAdxWvCcitgbGAdsAewLnRETP4jjnAhOAEcWyZ9F+GDA/M7cCzgROa+v3YSiSJEkdIiL6AB8GzgPIzMWZuQAYA0wqNpsE7Fu8HgNcnpmLMvNpYDowMiKGAH0y867MTOCiJvssP9YUYFS08bI6Q5EkSRUVEe1aWmFL4Hnggoh4MCJ+FREbAIMzcxZA8XNQsf1Q4Nm6/WcUbUOL103bS/tkZiOwEBjYlu/DUCRJUkX1aOcSERMi4v66ZUKTUzQAOwDnZub2wCsUQ2Ur0VzSyhbaW9rnTfPqM0mSKqq9N2/MzInAxBY2mQHMyMx7ivdTqIWiORExJDNnFUNjc+u236xu/2HAzKJ9WDPt9fvMiIgGoC8wry2fx0qRJEkV1dFXn2XmbODZiHhH0TQKeByYCowv2sYD1xavpwLjiivKhlObUH1vMcT2UkTsXMwXOqTJPsuPNRa4vZh39KZZKZIkSR3pKODSiFgX+AfwOWpFmckRcRjwDLA/QGY+FhGTqQWnRuCIzFxaHOdw4EKgN3BTsUBtEvfFETGdWoVoXFs7Gm0MUx3u5UVraMekbm7eK4u7ugtSZW0+oFen3k1xykOz2vW3duy7h3Sruz9aKZIkqaKcQ1NmKJIkqaLaO9G6uzEkSpIkYaVIkqTKsk5UZiiSJKmiHD0rMxRJklRRPawVlRiKJEmqKCtFZU60liRJwkqRJEmVFQ6flRiKJEmqKIfPygxFkiRVlBOtywxFkiRVlJWiMidaS5IkYaVIkqTKslJUZiiSJKmivPqszFAkSVJF9TATlTinSJIkCStFkiRVlsNnZYYiSZIqyonWZYYiSZIqykpRmaFIkqSKcqJ1mROtJUmSsFKkwqUXX8g1V00hCLYaMYKTTvlvevXqVdrm/vvu4YzT/5vGxkb69evHLy+4pF3nXLx4Md858Rs88fhj9O3bj1N/8CPeMnQYs2Y+x9e+chTLli2jsbGRAw78LGM/Pa5d55LWRHPnzOb0k09k3gv/okePHnxszKfY74DPlrb5052/48KJPyV69KBnz558+divs+27d2jXeRcvXszpJ5/IU399nD59+3Lif/6ATYcMfWP9K6+8zGHj9uUD/74bR33tm+06l9ZsDp+VWSkSc+fM4fJLL+biy6Yw+errWLpsGbfcfENpm5defJFTv38yPzr7HK68+npO++GPW338mc/NYMKhB6/Qfs1VU+jTpw/X3nArBx08nrPPOgOAjTfZhAsuvpzLrryGSZdewYXnT+T5uXPa9yGlNVDPnj354tHHcf7l13L2Ly9h6m+u4H+f/ntpm+3fuxO/uHgKv7joSr524sn86L++2+rjz571HMd9+dAV2m++7io23KgPk6bcwH7jDuZXPzurtP7CiT9lu+13bMtH0lomon1Ld2MoEgBLly5l0aLXaWxs5PXXX2OTTQaV1t904/XsNmp3hgx5CwADBg58Y92N10/lkM/sz4H778v3T/4OS5cubdU5f3/Hbey9z74AjNp9NPfecxeZyTrrrMu6664L1P5Fu2xZroZPKK15Bm68CSPesTUA62+wAZtvMZx/PT+3tE3v9dcnir8+r7/2Wukv0W9vvp4jD/0MXzxkf8469eRW/+796Q93sMfH9gHgwx/ZnQfvv4fM2u/Z3/76OAvmzWPHnd7f3o+ntUC0c+luOj0URcTnOvucatmgwYP57PhD+fgeuzF61IfYcMON2OX9Hyxt88z//pMXX3yRCYcezEEH7Mf1U68B4Ol//J1bb76R8yb9msuuvIYePXpy0w3Xteq8z8+Zy+DBQwBoaGhgww03YsGCBQDMnj2LAz61Dx/b4yP8x6GfZ5NBg1fb55XWRLNnPcf0v/2Vd27zbyus++Mdt3HoAfvwreOO4GsnngzA//7zH/z+tzdz1sRJ/OKiK+nRswe333LDCvs254Xn57DJ4NrvVM+GBjbYcENeXLiAZcuW8Yuzf8gXjvzq6vtgWqP1iGjX0t10xZyi7wEXNLciIiYAEwB+/NOfc+jnJ3RmvyrrxRcX8vvf3cZ1N/2WDTfaiG987VhuvH4qH9t7nze2Wbq0kScef4yf//ICXl+0iM8dPI5/2+7d3HvPXTzxxGMc8pn9AVj0+usMGDAAgOOOPZKZz81gyZIlzJ41iwP33xeAAw86mH32/RTJihWg5b9jm246hCt+M5Xn587huGOPZNTuoxk4cOOO/SKkLvLaq69y8glf5fBjv84GG2y4wvoP7jqKD+46iocfvJ8LJ/6U03/ySx687x7+9uQTHHHoZwBYvOh1+vWv/e599xvHMmvWczQuWcLcObP44iG1389Pfvog9tx7X7KZ4mtEMPU3VzDy/R9k0OBNO+7DSmuwDglFEfHwylYBK/0nf2ZOBCYCvLyouV9bdYR77r6LocOG0b8IM7uN2p2H/vJgKRQNGrwp/fr1p/f669N7/fXZYcf38re/PUlmsvc++3LUMcetcNwzzvopUJtT9N1vn8DE8y8urR80eDBz5sxi8Kab0tjYyMsvv0Tfvv1K22wyaDBbvm0rHvzz/Xx0jz1X8yeXul5j4xK+982vstvoj/OhXT/a4rbbbf9eZj33LRYumA8ke+y1D4d9+ZgVtvvuaWcBterTD075Nmecc35p/caDBvP8nDlsMmhTljY28srLL7NRn7488ehDPPLQA1z3m8m89tqrNC5ZQu/11+fzXz52NX1arWm6X62nfTpq+GwwcAjwiWaWFzronGqjTTcdwiMPP8Rrr71GZnLvPXcxfMstS9vs+pFRPPjAn2lsbOS1117j0YcfZvjwLRm50y7cNu1W5r1Q+7914cIFzJr5XKvO+++77vbGMNxt027hfSN3JiKYM3s2r7/+OlCrYj30lwd46xbDV98HltYQmckZ3z+Jzd86nLEHHtLsNs89+8wb832eevJxlixppE/ffmz/3p2483fTmD+v9rv34sKFzJk1s1Xn3eWDu3LrjVMBuPN303jPjiOJCE743qn8+ppbueTqm5lw1HF8dK9PGIi6OycVlXTU8Nn1wIaZ+ZemKyLijg46p9ro37Z7N6M+ugcHHbAfDT0beMe73sV+Yw9gyuTLARj76XEM3/JtvP8DH2Lc2DH0iB7su99YthrxdgC+fOQxHPGlw1i2bBkNDQ0c/83vMOQtQ1s6JQBjPjmWb3/z64z5+B707duX/zr9RwA8/fTfOfOHpxERZCYHjz+UEW9/R8d9AVIXeezhB/ntzdcz/G0j3hjiOvRLRzN39iwAPrHfp/nDHb/ltzddR8+GBnr16sW3/vN0IoK3Dn8bn/vikRx/7JfI4nfvyK99k8HFxRAt2esTn+TU732T8WM/zkZ9+nLiKad36OfUmstL8ssi19BRKofPpK4x75XFXd0FqbI2H9CrU1PKPX9f2K6/tTu9rW+3SlXevFGSpIrqhheQtYuhSJKkijITlXnzRkmSqqoTJlpHRM+IeDAiri/eD4iIaRHxVPGzf922J0TE9Ih4MiJG17XvGBGPFOvOjuKOphHRKyKuKNrviYgt2vN1GIokSaqoaOf/WukY4Im698cDt2XmCOC24j0RsTUwDtgG2BM4JyJ6FvucS+0+hiOKZfk9Wg4D5mfmVsCZwGlt/S7AUCRJkjpIRAwDPg78qq55DDCpeD0J2Leu/fLMXJSZTwPTgZERMQTok5l3Ze3qsIua7LP8WFOAUcurSG1hKJIkqaLa+0DYiJgQEffXLU0fRXEW8HVgWV3b4MycBVD8XP6wzaHAs3XbzSjahhavm7aX9snMRmAhMJA2cqK1JEkV1d6J1vVPoljh2BF7A3Mz888RsWsbu5MttLe0T5sYiiRJqqqOvfzsA8A+EfExYD2gT0RcAsyJiCGZOasYGptbbD8D2Kxu/2HAzKJ9WDPt9fvMiIgGoC8wr60ddvhMkqSK6siJ1pl5QmYOy8wtqE2gvj0zPwtMBcYXm40Hri1eTwXGFVeUDac2ofreYojtpYjYuZgvdEiTfZYfa2xxDitFkiRprXAqMDkiDgOeAfYHyMzHImIy8DjQCByRmUuLfQ4HLgR6AzcVC8B5wMURMZ1ahWhcezrmYz4klfiYD6nrdPZjPv7yzEvt+lv7ns036lb3f7RSJElSRXWrRLMaGIokSaoqU1GJE60lSZKwUiRJUmW9iUd1VIKhSJKkimr7AzG6J0ORJEkVZSYqMxRJklRVpqISJ1pLkiRhpUiSpMpyonWZoUiSpIpyonWZoUiSpIoyE5U5p0iSJAkrRZIkVZelohJDkSRJFeVE6zJDkSRJFeVE6zJDkSRJFWUmKnOitSRJElaKJEmqLktFJYYiSZIqyonWZYYiSZIqyonWZYYiSZIqykxU5kRrSZIkrBRJklRdlopKDEWSJFWUE63LDEWSJFWUE63LnFMkSZKElSJJkirLQlGZoUiSpKoyFZUYiiRJqignWpcZiiRJqignWpc50VqSJAkrRZIkVZaFojIrRZIkVVRE+5ZVHz82i4jfRcQTEfFYRBxTtA+IiGkR8VTxs3/dPidExPSIeDIiRte17xgRjxTrzo6o9SAiekXEFUX7PRGxRVu/D0ORJEmVFe1cVqkROC4z3wXsDBwREVsDxwO3ZeYI4LbiPcW6ccA2wJ7AORHRszjWucAEYESx7Fm0HwbMz8ytgDOB09rwRQCGIkmSKqujK0WZOSszHyhevwQ8AQwFxgCTis0mAfsWr8cAl2fmosx8GpgOjIyIIUCfzLwrMxO4qMk+y481BRi1vIr0ZhmKJElSm0TEhIi4v26Z0MK2WwDbA/cAgzNzFtSCEzCo2Gwo8GzdbjOKtqHF66btpX0ysxFYCAxsy+dxorUkSRXV3onWmTkRmLjK80RsCPwGODYzX2yhkNPcimyhvaV93jQrRZIkVVRHD5/VzhHrUAtEl2bmVUXznGJIjOLn3KJ9BrBZ3e7DgJlF+7Bm2kv7REQD0BeY9+a+iRpDkSRJFRXt/N8qj18rCZ0HPJGZP6pbNRUYX7weD1xb1z6uuKJsOLUJ1fcWQ2wvRcTOxTEPabLP8mONBW4v5h29aQ6fSZKkjvIB4GDgkYj4S9H2TeBUYHJEHAY8A+wPkJmPRcRk4HFqV64dkZlLi/0OBy4EegM3FQvUQtfFETGdWoVoXFs7G20MUx3u5UVraMekbm7eK4u7ugtSZW0+oFen3k9x9otL2vW3dtM+63Sr+z9aKZIkqaK6VaJZDQxFkiRVlA+ELTMUSZJUUa2ZLF0lXn0mSZKElSJJkqrLQlGJoUiSpIoyE5UZiiRJqignWpcZiiRJqignWpc50VqSJAkrRZIkVZbDZ2VWiiRJkrBSJElSZVkpKrNSJEmShJUiSZIqy6vPygxFkiRVlMNnZYYiSZIqykxUZiiSJKmqTEUlTrSWJEnCSpEkSZXlROsyQ5EkSRXlROsyQ5EkSRVlJiozFEmSVFWmohInWkuSJGGlSJKkynKidZmhSJKkinKidVlkZlf3Qd1QREzIzIld3Q+pavzdk9rOOUXqKBO6ugNSRfm7J7WRoUiSJAlDkSRJEmAoUsdxToPUNfzdk9rIidaSJElYKZIkSQIMRVrNImLPiHgyIqZHxPFd3R+pKiLi/IiYGxGPdnVfpLWVoUirTUT0BH4G7AVsDRwYEVt3ba+kyrgQ2LOrOyGtzQxFWp1GAtMz8x+ZuRi4HBjTxX2SKiEz7wTmdXU/pLWZoUir01Dg2br3M4o2SZLWeIYirU7NPUXHyxslSWsFQ5FWpxnAZnXvhwEzu6gvkiS9KYYirU73ASMiYnhErAuMA6Z2cZ8kSWoVQ5FWm8xsBI4EbgGeACZn5mNd2yupGiLiMuAu4B0RMSMiDuvqPklrG+9oLUmShJUiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRVKXiYilEfGXiHg0Iq6MiPXbcawLI2Js8fpXLT2INyJ2jYj3t+Ec/4yIjVvb3mSbl9/kub4bEV97s32UpPYwFEld57XMfE9mbgssBr5UvzIierbloJn5+cx8vIVNdgXedCiSpO7OUCStGf4AbFVUcX4XEb8GHomInhHxg4i4LyIejogvAkTNTyPi8Yi4ARi0/EARcUdEvLd4vWdEPBARD0XEbRGxBbXw9ZWiSvWhiNgkIn5TnOO+iPhAse/AiLg1Ih6MiF/Q/LPtSiLimoj4c0Q8FhETmqw7o+jLbRGxSdH2toi4udjnDxHxzmaOeXTxOR+OiMvb+P1K0io1dHUHpKqLiAZgL+DmomkksG1mPl0Ei4WZ+b6I6AX8T0TcCmwPvAP4N2Aw8DhwfpPjbgL8EvhwcawBmTkvIn4OvJyZPyy2+zVwZmb+MSI2p3ZH8ncBJwF/zMyTI+LjQCnkrMShxTl6A/dFxG8y8wVgA+CBzDwuIr5THPtIYCLwpcx8KiJ2As4BdmtyzOOB4Zm5KCL6teY7laS2MBRJXad3RPyleP0H4Dxqw1r3ZubTRfsewHbL5wsBfYERwIeByzJzKTAzIm5v5vg7A3cuP1ZmzltJPz4KbB3xRiGoT0RsVJxjv2LfGyJifis+09ER8cni9WZFX18AlgFXFO2XAFdFxIbF572y7ty9mjnmw8ClEXENcE0r+iBJbWIokrrOa5n5nvqGIhy8Ut8EHJWZtzTZ7mPAqp7RE63YBmrD6Ltk5mvN9KXVzwGKiF2pBaxdMvPViLgDWG8lm2dx3gVNv4NmfJxaQNsH+HZEbFM8Z0+SVivnFElrtluAwyNiHYCIeHtEbADcCYwr5hwNAT7SzL53Af8eEcOLfQcU7S8BG9Vtdyu1oSyK7d5TvLwTOKho2wvov4q+9gXmF4HondQqVcv1AJZXuz5DbVjuReDpiNi/OEdExLvrDxgRPYDNMvN3wNeBfsCGq+iHJLWJlSJpzfYrYAvggaiVbp4H9gWupjb35hHgb8Dvm+6Ymc8Xc5KuKsLFXGB34DpgSkSMAY4CjgZ+FhEPU/tvwp3UJmN/D7gsIh4ojv/MKvp6M/Cl4jhPAnfXrXsF2CYi/gwsBA4o2g8Czo2IbwHrAJcDD9Xt1xO4JCL6Uqt8nZmZC1bRD0lqk8hsdXVckiSp23L4TJIkCUORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBMD/B+e9t0RLWDzdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "y_pred=np.array(y_pred)\n",
    "confusion_matrix_result=confusion_matrix(y,y_pred)\n",
    "plt.figure(figsize=(10,6))\n",
    "sns.heatmap(confusion_matrix_result,annot=True,cmap='Blues')\n",
    "plt.xlabel('Predicted labels')\n",
    "plt.ylabel('True labels')\n",
    "plt.show()\n",
    "#混淆矩阵将错误数据可视化\n",
    "#相较于神经网络，随机森林的分类错误数据变少了，准确率提升了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### AUC方法检验模型的可用性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABFiklEQVR4nO3dd3gU5fbA8e9JQkiA0EJROkgJ5QJKBylSpIjtZ8Fy8eLVK6HZUPFiwy4WEKWJqHht2AFBRbEAotIkdAREhCC9hJIEUs7vjxnCElI2kM3uJufzPPvs9Dk7uztn3ndm3hFVxRhjjMlOiL8DMMYYE9gsURhjjMmRJQpjjDE5skRhjDEmR5YojDHG5MgShTHGmBxZoigkRGStiHTxdxz+JiKTReSRAl7nNBF5qiDX6SsicrOIfHOW8xba36CIqIjU9Xcc/iJ2H0X+E5GtQGUgDTgKfA0MVdWj/oyrsBGRAcDtqnqxn+OYBsSr6sN+jmMUUFdV/1kA65pGAHzmgiIiCtRT1c3+jsUfrEThO5eraimgOXAh8F//hpN3IhJWFNftT7bNTUBSVXvl8wvYCnT36H8emOPR3xb4GTgErAS6eIwrD7wF/A0cBGZ4jOsLxLnz/Qw0zbxOoAqQBJT3GHchsA8o5vb/G1jvLn8uUNNjWgWGAJuAP7P5fFcAa904fgQaZorjv8A6d/lvARF5+AwjgFXAcSAMeBD4AzjiLvNqd9qGQDKnSm2H3OHTgKfc7i5APDAc2APsBG71WF808AVwGFgKPAX8lMP3erHH97YdGOCxzgnAHDfOxcAFHvONc6c/DCwHOnqMGwV8Arzrjr8daA384q5nJzAeCPeYpzHwLXAA2A2MBHoBJ4AUd3usdKctA7zhLmeH+xlD3XEDgEXAWHdZT7nDfnLHiztuD5Dgfi9NgDvc9Zxw1/VF5t89EOrGdfK7Ww5Uz2a7Zvl/ANrj/G6ru/3N3Gli3P4sfxtZfLZDwBZ3eQPc72IP8C+P6acBk93tegSYz5n/i7pud3HgRWCbu/0nA5H+3u/4dJ/m7wAK4yvTH6YasBoY5/ZXBfYDfXBKdD3c/oru+DnAh0A5oBjQ2R1+kfvjbuP+Cf/lrqd4Fuv8HviPRzwvAJPd7quAzTg72jDgYeBnj2nV/bOUz+rHD9QHjrlxFwMecJcX7hHHGqC6u4xFnNpxe/MZ4tx5I91h1+EkvxCgn7vu891xA8i0Y+fMRJEKPOHG2gdIBMq546e7rxJAI5wdSJaJAqiBswO50V1WNNDcY50HcHbwYcB7wHSPef/pTh+Gk7R24SZPnESR4n4vIUAk0AJn5xkG1MJJ6ne700fh7PSHAxFufxuPZb2bKe4ZwGtASaASsAQY6LH9UoFh7roiOT1R9MTZwZfFSRoNPbZ9xnbO5nd/P87vvoE7bzMgOovtmtv/4Wmc33MkTqIa6jFvbr+NVOBWnN/aUzg79gk4O/pL3e+zlMfnOQJ0csePw+O3wOmJ4mVgFs7vOwrnYONZf+93fLpP83cAhfHl/mGOuj88Bb4DyrrjRgDvZJp+Ls5O83wgHXdHlmmaScCTmYb9zqlE4vknvR343u0WnB1gJ7f/K+A2j2WE4Ow8a7r9CnTN4bM9AnyUaf4dnDoK3ArEeozvA/yRh8/w71y2bRxwpds9gNwTRRIQ5jF+D85OOBRnB93AY1y2JQqcUtLn2YybBkzN9Jk35PAZDgLN3O5RwIJcPvPdJ9eNk6hWZDPdKDwSBc55suN4JHx3/h88tt+2TMvI2KZAV2Cju71CstvOmX73J3+Dv5/8nnL5bNn+H9zuYjjJajXOuT7Jw29jk8e4f+D8tit7DNvP6cneM7mXwimtnizNKFAX5/90jNNLjO3IpvRdWF52jsJ3rlLVKJydVQxQwR1eE7hORA6dfOFUaZyPcyR9QFUPZrG8msDwTPNVxzmiyuwToJ2IVME5QlJgocdyxnks4wDOj7+qx/zbc/hcVYC/Tvaoaro7fXbz/+URozef4bR1i8gtIhLnMX0TTm1Lb+xX1VSP/kScnUBFnKNoz/Xl9Lmr41RzZGdXFusAQESGi8h6EUlwP0MZTv8MmT9zfRGZLSK7ROQw8IzH9LnF4akmzo52p8f2ew2nZJHluj2p6vc41V4TgN0iMkVESnu5bm/jzOn/gKqm4OzEmwAvqbtnBq9+G7s9upPc5WUeVsqjP2NbqHPhyQHO/H9VxCmBLvdY79fu8ELLEoWPqep8nB/6i+6g7ThHUGU9XiVV9Tl3XHkRKZvForYDT2ear4SqfpDFOg8B3wDXAzcBH3j8wbbjVD14LidSVX/2XEQOH+lvnD83ACIiODuFHR7TVPforuHO4+1n8NwR1AReB4biVFuUxanWEi/izM1enKqJatnEndl24IK8rkREOuIcNV+PU1Isi1PfLx6TZf4ck4ANOFfZlMap6z85fU5xZF7OdpwSRQWP7V1aVRvnMM/pC1R9RVVb4JwXqY9TpZTrfLnEmXm67P4PiEhV4DGcc10viUhxd3huv42zkfH9i0gpnKqlvzNNsw8nwTT2iLeMOheuFFqWKArGy0APEWmOc9LychHpKSKhIhIhIl1EpJqq7sSpGpooIuVEpJiIdHKX8ToQKyJtxFFSRC4Tkahs1vk+cAtwjdt90mTgvyLSGEBEyojIdXn4LB8Bl4lINxEphlNXfhznZORJQ0SkmoiUx9nJfXiWn6Ekzg5prxvrrThHjSftBqqJSHge4gdAVdOAz4BRIlJCRGJwtld23gO6i8j1IhImItHu95mbKJyEtBcIE5FHgdyOyqNwTmwfdeMa5DFuNnCeiNwtIsVFJEpE2rjjdgO1RCTE/Yw7cQ4YXhKR0iISIiIXiEhnL+JGRFq531UxnOqWkxcPnFxXnRxmnwo8KSL13O+6qYhEZzFdtv8H9yBkGs7J+Ntwzs086c6X22/jbPQRkYvd39OTwGJVPa3E5ZagXwfGikgld91VRaTnOa47oFmiKACquhf4H/CI+8O7EmcHuhfniOp+Tn0X/XHqzjfg1Kff7S5jGfAfnKqAgzgnkAfksNpZQD1gt6qu9Ijlc2A0MN2t1lgD9M7DZ/kd5+TsqzhHV5fjXAp8wmOy93F2UFvc11Nn8xlUdR3wEs4VQLtx6pkXeUzyPc7VV7tEZJ+3n8HDUJxqoF3AO8AHOEkvq1i24Zx7GI5TJRGHc4I2N3Nxkv9GnGq4ZHKu4gK4D6ckeARnp3Qy0aKqR3BO+F7uxr0JuMQd/bH7vl9EfnO7bwHCOXUV2ie41TpeKO2u/6Ab+35OlYzfABq51S8zsph3DM5BxTc4Se8NnBPSp8nl/3AnznmWR9wS8a3ArSLS0Yvfxtl4H6f0cgDngoKbs5luBM5v91f3PzQP56R9oWU33Jl8Jc7Nhrer6jx/x5JXIjIaOE9V/+XvWEzBkiJ2A2FeWYnCFFkiEuNWiYiItMap3vjc33EZE2jsTkxTlEXhVDdVwanmewmY6deIjAlAVvVkjDEmR1b1ZIwxJkdBV/VUoUIFrVWrlr/DMMaYoLJ8+fJ9qnpWNwYGXaKoVasWy5Yt83cYxhgTVETkr9ynyppVPRljjMmRJQpjjDE5skRhjDEmR5YojDHG5MgShTHGmBxZojDGGJMjnyUKEXlTRPaIyJpsxouIvCIim0VklYhc5KtYjDHGnD1f3kcxDac56f9lM743TjPY9XCeoTzJfTfGO6qg6ae/SD9zmKZD+smH3OmpebPqzm28ekyXXXdeptVM6z1tGJnGeXzuMzdGLtPk0lRPjk35+HDenMbn2ryQj+bNdf7g2x4nTqTnMm/OfJYoVHWBiNTKYZIrgf+57cz/KiJlReR892ErJlClJEHSvlOvlCOQmuy+kiDNfc88LOXYqVfaCUg/4b6nONOkp7o79TSPHbxHd3oakO4s0xjjtXEL2zB18blV2PjzzuyqnP4Al3h32BmJQkTuAO4AqFGjRoEEV+SkJMKhP2DvSkjcDcd2ebzvgeQDkLQfUhP9HenpJOT0FyFnDpMQQCBpL5Q8j4ynZYqc6kbcfrIef0Z3VvPkYdps58s8Ho/pPHuzeuJnbtPk9pTQHMZnub5CPG+u8wfPZ2p2tDTrvqiUxbTe82eiyOoTZ1l2UtUpwBSAli1bWnO35yJpP+xfDwfc18nuw17e3R9SDEpUhMiKEBkN4aUhLBJCIyAswukOizi9PzQCipU89QotDqHhzrJCw53pQsIz7dxDs9jhu8NCQt3uc3k8sjGF0/btCcyevZFBg1oB0OUa2HzLQerUefysl+nPRBHP6Q+zr8aZDzI35+LoTvh7Efz9K+xfC7uXOdVFWQkJg1LVoMI/oGwdKHGec/RdsjKUqAQR0RBRHsKjbAdtTABKTU3nlVcW8+ijP3DsWApNmlSiY8eaANSuXe6clu3PRDELGCoi03FOYifY+YlzcHgb7FgIBzbC4a1Ogjj0x5nTFSsJ5WOgfEOIbui8l28IZS+A0GIFHrYx5twtXhzPwIGzWblyNwDXXNOQOnXOLTl48lmiEJEPgC5ABRGJx3loeTEAVZ0MfInzsPrNQCLOg9ONt1Th719g02fw1zewb/WZ0xQrBVXaQZX2UKEJnNcKompYicCYQuLgwSRGjvyO115bjirUqlWW8eN7c9ll9fN1Pb686unGXMYrMMRX6y+00lOd5LB8DOxcfGp4sVJQvTNUbA6la0GFxk5iCAm6luSNMV56/PH5TJ68nLCwEO67rx2PPNKZEiXyv2bA9iLB4vhhWPMG/Dbu1InniPLQ5N9QswdU6wxhxf0bozHG51JT0wkLc+6VfvjhTvz55yGefrorTZqc25VNObFEEehOHIGlz8Nvr8CJw86wcvXgonug8S3OOQdjTKGXnJzK6NE/MWPG7yxefDvh4aFUqFCCmTNv8Pm6LVEEqrQUpwTx82POfQzglBpa3AsX9HXvDTDGFAXffbeFQYPmsGnTAQDmzt3M5Zc3KLD1W6IIJIl7Ye002PET/P3zqUtZq7SHTi9A1fZ+Dc8YU7B27z7K8OHf8N57zsUqDRtWYNKky+jcuVaBxmGJIhAk7oVlL8KK8aff+VyuHlz8DNS7xq5UMqaIeffdVQwb9hWHDiUTERHGo492Yvjw9oSHhxZ4LJYo/Clxn5Mg4sY7bSAB1O4DMTfA+e2cexssQRhTJKWnK4cOJdOrV10mTOiTr/dF5JUlCn9I3AfLX4IVr55KEHUug3aPOZe0GmOKnKNHT/DLL9vp0eMCAPr3b0qVKlF061Yb8fMBoyWKgpS417n/wTNB1O7jJIjzW/s3NmOM38yYsYFhw75i795jrFkzmLp1yyMidO9ex9+hAZYoCsaRHfDL486J6vQUZ1jt3tBulCUIY4qwv/46xJ13fs2sWb8D0LJlFY4fT81lroJnicKXkg/CktGwYpzzHAUJgTp9oe3DcL49o8mYoiolJY2XX/6VUaPmk5iYQlRUOM88041Bg1oSGhp4l75bovCFlGPwyxNOFVNqkjOs3jVw8dNQvuCufTbGBKY77/yKyZOXA3D99Y0ZO7YnVapE+Tmq7FmiyG9H4uHDzpCwxemv2QM6PGVVTMaYDHff3Zb58/9izJie9OpV19/h5MoSRX5RhfXvwvfD4HgClKsPvf9nVUzGFHGqyrvvruLLLzfz/vv/h4jQoEEF1qwZTEhIcFz+bokiPyQfhDk3wdavnf5qneHyj5wH/hhjiqzff9/HoEFz+OGHrYBzyWufPvUAgiZJgCWKc7d/Hcy4Eg5tdp4Cd/FT0HSg3ShnTBGWlJTCs8/+xOjRizhxIo3o6EheeulSevcO/GqmrFiiOFtpJ2DJc7D0BUg56jwH4qoZULqmvyMzxvjRvHlbiI2dzR9/HATgttsuZPTo7kRHl/BzZGfPEsXZ2LsK5t8Hf33r9De4AXq+AcWC94dgjMkfP/+8nT/+OEjjxhWZPLkvF19cw98hnTNLFHm18jWYF+t0R0RD3+lQo5tVNRlTRKWlpbN58wEaNKgAwIgRHahQoQS3336RXxrw84XAu7MjkK1+41SSaBYL/ZdDze6WJIwpolas2En79m9y8cVvceCAc89U8eJhDB7cqtAkCbBE4b3t8+HbO5zuzi9B90l2PsKYIurIkePcc8/XtGz5OkuW7KB48VD++OOAv8PyGat68kbiXvjyJtB0aDUCWt7r74iMMX6gqnz22Xruuutrduw4QkiIcM89bXn88S5ERRXeZ9ZboshN6nGY9X9w9G+o0sG5/NUYUyTdfffXvPLKEgBatarCa6/15cILz/dzVL5nVU85UYWv+juPJi1VFfp+CCGWW40pqq6+uiFlyhRnwoQ+/PLLbUUiSYCVKHK27EXY+DEUKwlXz4aoqv6OyBhTgH76aRs//PAnjzzSGYAuXWqxbds9lC5deKuZsmKJIjuHt8Oih53uPu9DpeZ+DccYU3D2709kxIh5vPHGCgC6datD+/bVAYpckgBLFFlThR/ucu6+btAP6l7h74iMMQVAVfnf/1Zy333fsm9fIsWKhfDggxdz4YXn+Ts0v7JEkVnSfvjmP7D5cyhWCjo97++IjDEFYP36vQwaNIf58/8C4JJLajFx4mXExFTwc2T+Z4kis/nDnSQBcMk4KB38t98bY3I3ZswvzJ//FxUrlmDMmJ7cfPM/ELuZFrBEcbqdi2Ht284jS//vS6jV098RGWN8KCEhmTJlIgB49tnulCwZzqOPdqZ8+Ug/RxZY7PLYkxK2wsyrnO4Wwy1JGFOI/f33Efr1+4S2bd/gxIk0ACpUKMHLL/eyJJEFSxQA6WnwxXVwbBdUvwQ6POnviIwxPpCWls6rry4mJmY8H320lm3bEvjtt53+DivgWdUTwJo3YfcyKFUNrvwcwore5W/GFHbLl//NwIGzWb7cSQxXXNGAV1/tTY0aZfwcWeDzaYlCRHqJyO8isllEHsxifBkR+UJEVorIWhG51ZfxZCklERY94nR3fhGK24/GmMJm1Kgfad16KsuX76R69dLMmNGPmTNvsCThJZ+VKEQkFJgA9ADigaUiMktV13lMNgRYp6qXi0hF4HcReU9VT/gqrjOseg0Sd0PlltDg+gJbrTGm4NSpUw4RGD68HaNGdaFUqXB/hxRUfFn11BrYrKpbAERkOnAl4JkoFIgS5xq0UsABINWHMZ0u6QD86p6PaPeoPVfCmEJiy5aDLF26g379mgDQv39T2rSpmvFwIZM3vkwUVYHtHv3xQJtM04wHZgF/A1FAP1VNz7wgEbkDuAOgRo18vK9h+RhIPgg1ukKdvvm3XGOMX5w4kcaLL/7Mk08uQFVp0aIKdeuWR0QsSZwDX56jyOrwXDP19wTigCpAc2C8iJQ+YybVKaraUlVbVqxYMX+iS9oPK15xuts/aaUJY4LcggV/0bz5ZB566HuSk1O59tpGRbJdJl/wZYkiHqju0V8Np+Tg6VbgOVVVYLOI/AnEAEt8GJdj+Rg4cQRqXgpV2/t8dcYY39i3L5H77/+WadPiAKhXrzyTJl1Gt251/BtYIeLLRLEUqCcitYEdwA3ATZmm2QZ0AxaKSGWgAbDFhzE5kvbDbydLE4/7fHXGGN+JjZ3Np5+up3jxUEaO7MgDD3QgIsKu/M9PPtuaqpoqIkOBuUAo8KaqrhWRWHf8ZOBJYJqIrMapqhqhqvt8FVOGJaMh5SjU6gVV2vp8dcaY/JWeroSEONXFTz/dlaSkVF5+uSf16kX7ObLCSZxan+DRsmVLXbZs2dkv4MgOeLMupCbDP5dB5Rb5F5wxxqcSE1N48sn5xMXt5ssvb7JG+/JARJarasuzmbfolc9+GukkiXrXWJIwJojMmbORoUO/YuvWQ4jAkiU7aNOmmr/DKhKKVqKIXwDr/gdhEXDxM/6Oxhjjhfj4w9x119d89tl6AJo1q8zkyX0tSRSgopMoNB1+uMfpbvUglK/v33iMMbmaOHEpI0bM4+jRE5QsWYwnn7yEYcPaEBZm7ZkWpKKTKP7+Ffb8BiXPg1b3+zsaY4wX9u1L5OjRE1x9dQzjxvWienVrm8kfik6i2Pix8x5zIxQr4d9YjDFZOnQomQ0b9tG2rVOtNGJEB1q3rkqvXnX9HFnRVjTKb0n7Ye1bTnfMjf6NxRhzBlVl+vQ1NGw4gSuu+IADB5IAKF48zJJEACgaiWLpC3A8AWp0h/Na+TsaY4yHzZsP0KvXe9x446fs2nWUevWiSUhI9ndYxkPhr3pKTztVmujwhH9jMcZkOH48leefX8TTTy/k+PE0ypWL4Pnne/Dvf1+YcTOdCQxeJwoRKamqx3wZjE/sXQWJeyCqOpxvd2EbEyj69fuEmTN/B+CWW5rxwgs9qFSppJ+jMlnJtepJRNqLyDpgvdvfTEQm+jyy/PL7dOe9dm9rIdaYAHL33W2JianA99/fwttvX2VJIoB5c45iLE5z4PsBVHUl0MmXQeUbTYf17zndDfv7NxZjirD0dGXq1N8YPnxuxrAuXWqxZs0gLrmkth8jM97wqupJVbdnalMlzTfh5LPt8+HoDihdy5oSN8ZPVq/eTWzsHH7+2XmO2S23NKNZs/MACA0tGtfTBDtvEsV2EWkPqIiEA3fiVkMFvLXTnPeGN4PYD9KYgnTs2Akef3w+Y8b8Qlqact55pXj55Z40bVrZ36GZPPImUcQC43AebRoPfAMM9mVQ+WLfWlj3DiDQ6BZ/R2NMkfLFF78zdOhXbNuWgAgMGdKKp5/uSpkyEf4OzZwFbxJFA1W92XOAiHQAFvkmpHyy6VNAofG/rF0nYwrYjBkb2LYtgQsvPI/XXutLq1ZV/R2SOQfeJIpXgYu8GBZY1r/rvDe4wb9xGFMEpKams2PHYWrWLAvA6NE9uPDC84mNbWkN+BUC2SYKEWkHtAcqisi9HqNK4zyxLnBtnw8HN0FEOajZw9/RGFOo/fprPLGxszl+PI2VK2MJDw+lQoUSDB3a2t+hmXySU6oPB0rhJJMoj9dh4Frfh3YO1r7tvFe/BEICO6cZE6wOHkxi0KDZtG//BitX7iY5OZWtWw/5OyzjA9mWKFR1PjBfRKap6l8FGNO5SU+DLbOd7jYP+zcWYwohVeWDD9Zwzz1z2bPnGGFhIdx/f3sefrgTJUoU83d4xge8OUeRKCIvAI2BjEsWVLWrz6I6F/vWQNJeKF0TKjX3dzTGFDo33/wZH3ywBoCOHWswadJlNG5cyc9RGV/y5izTe8AGoDbwOLAVWOrDmM7NrsXOe5X21mSHMT7Qq1ddoqMjefPNK/jxxwGWJIoAb0oU0ar6hojc5VEdNd/XgZ217W5o1gCgMfli3rwt/PHHAQYObAlA//5N6du3PuXLR/o5MlNQvEkUKe77ThG5DPgbCNynmu/81Xmv3sWvYRgT7HbvPsq9937D+++vpnjxULp3r8MFF5RHRCxJFDHeJIqnRKQMMBzn/onSwN2+DOqsJR2AhC0QFgnRjfwdjTFBKT1dmTJlOQ8+OI+EhONERITx6KOd7HnVRViuiUJV3UuISAAugYw7swPP7uXOe8VmEFL4n8lkTH5buXIXAwfOZvHiHQD07l2X8eP7UKdOOT9HZvwppxvuQoHrcdp4+lpV14hIX2AkEAlcWDAh5sHOX5z389v4Nw5jgtQDD8xj8eIdVKkSxbhxvbjmmoaIXRRS5OV02P0GUB1YArwiIn8B7YAHVXVGAcSWd7t/c94rt/BvHMYECVUlMTGFkiXDAXjllV5MnryMxx+/hNKli/s5OhMockoULYGmqpouIhHAPqCuqu4qmNDOwm73ql274smYXP311yGGDfuKY8dSmDevPyJCgwYVGDu2l79DMwEmp0RxQlXTAVQ1WUQ2BnSSOLYLjv4N4aWh7AX+jsaYgJWSksbYsb/y+OPzSUxMISoqnE2bDlC/frS/QzMBKqdEESMiq9xuAS5w+wVQVW3q8+jyYs8K573ShfaQImOysWjRNmJj57BmzR4A+vVrzJgxPalSJcrPkZlAllOiaFhgUeSHjPMTgd36uTH+MmzYl4wf71TP1qlTjgkT+tCrV10/R2WCQU6NAgZPQ4AAe9xEUSnwLsYyJhBUrFiSYsVCGDGiAyNHdiQy0hrwM97xaR2NiPQSkd9FZLOIPJjNNF1EJE5E1p5T0yAnSxSVrERhDMCGDfv45ps/MvpHjOjAqlWDePLJrpYkTJ747K409z6MCUAPnGdtLxWRWaq6zmOassBEoJeqbhORs2tdLPkgHN7q3JFdvsE5x25MMEtKSuGZZxYyevQiypaNYMOGoZQvH0nx4mHExFTwd3gmCHmVKEQkEqihqr/nYdmtgc2qusVdxnTgSmCdxzQ3AZ+p6jYAVd2Th+WfcvJEtt2RbYq4b775g8GD5/DHHwcBuOKKBtaIsjlnuVY9icjlQBzwtdvfXERmebHsqsB2j/54d5in+kA5EflRRJaLyC1eRZ2ZVTuZIm7nziPccMMn9Oz5Ln/8cZDGjSuycOGtTJ16BeXKWQN+5tx4c/g9Cqd08COAqsaJSC0v5svqOEazWH8LoBtOsyC/iMivqrrxtAWJ3AHcAVCjRo0zl7o3znm3E9mmiPq///uIX3+NJzIyjFGjunDPPW0pVsweA2zyhzcns1NVNeEslh2P0wTISdVwmijPPM3XqnpMVfcBC4BmmRekqlNUtaWqtqxYseKZazqwwXkvH3MWYRoTnFRPHXc991w3+vatz7p1Q3jggQ6WJEy+8iZRrBGRm4BQEaknIq8CP3sx31KgnojUFpFw4AYgc5XVTKCjiISJSAmgDbA+D/GDqiUKU6QcOXKce+75moEDZ2cM69y5Fl98cSO1apX1X2Cm0PImUQzDeV72ceB9nObG785tJlVNBYYCc3F2/h+p6loRiRWRWHea9TjnPlbhND44VVXX5OkTJO2HlGNO0x0l7IoOU3ipKp9+uo6GDSfw8suLeeutOLZuPeTvsEwR4M05igaq+hDwUF4XrqpfAl9mGjY5U/8LwAt5XXaGI+59gaVrnvUijAl0f/55kKFDv+LLLzcB0Lp1VSZPvsxKEKZAeJMoxojI+cDHwHRVXevjmPLmsCUKU3ipKs8/v4jHH59PUlIqZcoU59lnu3HHHS0IDbU2zUzB8OYJd5eIyHk4DzGaIiKlgQ9V9SmfR+cNSxSmEBMRNm7cT1JSKjfe2IQxY3py3nml/B2WKWK8OiRR1V2q+goQi3NPxaO+DCpPLFGYQmbfvsSM1l0BRo/uwTff/JP337/GkoTxC29uuGsoIqNEZA0wHueKp2o+j8xblihMIaGqTJsWR0zMeK677mNOnEgDoEKFEvToYc9YMf7jzTmKt4APgEtVNfN9EP5nicIUAuvX7yU2dg4LFji/52bNzuPgwSQqV7YShPE/b85RBPZzRS1RmCCWmJjC008v4IUXfiYlJZ2KFUswZkxPbr75H4g10mQCRLaJQkQ+UtXrRWQ1pze9EThPuDtxFJIPQGhxKHF2Dc8a4y+qSteub7N48Q4ABg5swbPPdrO2mUzAyalEcZf73rcgAjkrR+Kd96jq9vhTE3REhMGDW5GYmMJrr/WlXbvquc9kjB9ku3dV1Z1u52BV/cvzBQwumPBycfRkogicc+vGZCctLZ1XX13MmDG/ZAzr378py5ffYUnCBDRvDsN7ZDGsd34HclZOlihKWaIwgW3Zsr9p02Yqd975NSNHfsfffx8BnFKFNeBnAl1O5ygG4ZQc6ojIKo9RUcAiXwfmFStRmACXkJDMww9/z4QJS1GF6tVL8+qrvalSJcrfoRnjtZzOUbwPfAU8C3g+7/qIqh7waVTeshKFCVCqyscfr+Puu79m586jhIYK99zTlsce60KpUuH+Ds+YPMkpUaiqbhWRIZlHiEj5gEgWVqIwAey115azc+dR2ratxuTJl9Gs2Xn+DsmYs5JbiaIvsBzn8ljPi7oVqOPDuLxzxBKFCRzHj6dy6FAylSuXQkSYOLEPP/64lf/8pwUhIXZPhAle2SYKVe3rvtcuuHDyyKqeTICYP38rsbFzqFIlinnz+iMiNGhQgQYN7BkpJvh509ZTBxEp6Xb/U0TGiEgWD64uYClJkLwfQopBiSwej2pMAdi79xgDBsygS5e32bBhH9u3J7B79zF/h2VMvvLm8thJQKKINAMeAP4C3vFpVN446tzNSqmqdrOdKXDp6cobb/xGTMwE3n57JcWLh/L4411YtWqQtfBqCh1vGgVMVVUVkSuBcar6hoj8y9eB5cpOZBs/UVV69nyXefO2ANC9ex0mTuxDvXrRfo7MGN/wJlEcEZH/Av2BjiISChTzbVhesPMTxk9EhI4da7B69W7Gju3JDTc0sQb8TKHmTZ1NP+A48G9V3QVU5VyecZ1fMhJFVf/GYYqEOXM2MmPGhoz+ESM6sGHDUG680Vp5NYWfN82M7xKR94BWItIXWKKq//N9aLlI3OW8l6ri3zhMoRYff5i77vqazz5bT4UKJejUqSbly0dSvHgYxYt7UyA3Jvh5c9XT9cAS4Dqc52YvFpFrfR1Yro65iaKk3cRk8l9qajpjx/5Cw4YT+Oyz9ZQsWYyRIy+mdOni/g7NmALnzSHRQ0ArVd0DICIVgXnAJ74MLFeWKIyPLFmyg4EDZxMX5/zGrr46hnHjelG9ehk/R2aMf3iTKEJOJgnXfrw7t+FbliiMD6SnK7feOpN16/ZSo0YZxo/vzeWXN/B3WMb4lTeJ4msRmYvz3GxwTm5/6buQvHTyHEUJSxTm3Kgqx4+nERERRkiIMGFCH776ahOPPtqZkiWtAT9jvDmZfb+I/B9wMU57T1NU9XOfR5aTlCQ4nuDclR1Rzq+hmOC2efMBBg+eQ/XqpXnjjSsB6NKlFl261PJvYMYEkJyeR1EPeBG4AFgN3KeqOwoqsBwl7nbeS54HdmmiOQvHj6cyevQinnlmIcePp1G+fCTPP59IdHQJf4dmTMDJ6VzDm8Bs4BqcFmRfLZCIvGHnJ8w5+P77P2nadDKPPfYjx4+n8a9/NWPDhiGWJIzJRk5VT1Gq+rrb/buI/FYQAXnlmJ2fMHmXlpbOrbfO5J13nAc2NmgQzeTJfa2ayZhc5JQoIkTkQk49hyLSs19V/Zc4Eq1EYfIuNDSEsLAQIiLCePjhjtx3X3u7ac4YL+T0L9kJjPHo3+XRr0BXXwWVK6t6Ml5avXo3ycmptGrlNPXywgs9eOihjlxwQXk/R2ZM8MjpwUWXFGQgeXJsp/NuicJk49ixE4wa9SNjx/5KvXrRrFwZS3h4KNHRJexchDF5FJzl7oMbnffStfwahglMs2b9zrBhX7FtWwIi0L17bVJS0ggPD/V3aMYEJZ/eYS0ivUTkdxHZLCIP5jBdKxFJ87oNqSPbnfeyF+RLnKZw2LYtgauums6VV05n27YELrrofJYs+Q+vvtrHbpwz5hz4rEThPrdiAtADiAeWisgsVV2XxXSjgbleLzxpn/MeaY9ANY60tHS6dJnGn38eIioqnKee6srgwa0IC/N/azPGBLtcE4U4je3fDNRR1Sfc52Wfp6pLcpm1NbBZVbe4y5kOXAmsyzTdMOBToJV3IatzV7aEQERZ72YxhZaqIiKEhoYwalQXvvhiIy+/3JOqVUv7OzRjCg1vDrcmAu2AG93+IzglhdxUBbZ79Me7wzKISFXgamByTgsSkTtEZJmILNu/122fMKK8PSu7CDt4MInY2Nk888zCjGH9+zfl44+vsyRhTD7zpuqpjapeJCIrAFT1oIh4U+GbVdsamqn/ZWCEqqbl9JQwVZ0CTAFo2byRswyrdiqSVJX331/Nvfd+w549x4iKCmfo0NaUKRNhT5ozxke8SRQp7nkEhYznUaR7MV88UN2jvxrwd6ZpWgLT3T94BaCPiKSq6oxsl5qa7LyXj/EiBFOYbNy4n8GD5/Ddd38C0LFjDSZNuowyZSL8HJkxhZs3ieIV4HOgkog8DVwLPOzFfEuBeiJSG9gB3ADc5DmBqtY+2S0i04DZOSYJgLQTznuZ2jlOZgqP1NR0nnpqAc8++xMnTqQRHR3JCy/0YMCA5laKMKYAeNPM+HsishzohlOddJWqrvdivlQRGYpzNVMo8KaqrhWRWHd8juclspWe6ryXqHxWs5vgExoqLFy4jRMn0vj3v5szenQPKlSwm+aMKSiimvm0QaYJnKuczqCq23wSUS5a1ovWZbEHoOdb0GSAP0IwBWD37qMkJ6dSs2ZZADZt2s/OnUfp1KmmfwMzJkiJyHJVbXk283pT9TQH5/yEABFAbeB3oPHZrPCcqVuiiIz2y+qNb6WnK1OmLOfBB+fRsmUVvv22PyJCvXrR1Ktn37kx/uBN1dM/PPtF5CJgoM8iyk16mvNe3B50X9jExe0iNnY2ixc7z8cKDw/l6NETREUV93NkxhRteb4zW1V/ExEvb47zAXUvuAq3a+ULiyNHjvPYYz8ybtxi0tOVKlWiGDeuF9dc09BOVhsTALy5M/tej94Q4CJgr88iyo26JQpLFIXCiRNpXHTRFDZvPkBIiHDXXW144olLKF3aShHGBApvShRRHt2pOOcsPvVNOF44WfUUHpXzdCYohIeH0r9/U774YiOTJ19GixZV/B2SMSaTHK96cm+0e05V7y+4kHLWsnqILrtb4a5kCLOjzmCTkpLG2LG/UqNGGW64oQnglCpCQ532mowxvuGTq55EJMy9F+Kisw/NFxRCwy1JBKFFi7YRGzuHNWv2ULFiCfr2rU+pUuH2nAhjAlxOVU9LcM5HxInILOBj4NjJkar6mY9jy16EPcYymBw4kMSIEd8ydeoKAOrUKcfEiX0oVcqeEWFMMPDmHEV5YD/OM7JP3k+hgP8SRYjtYIKBqvLOO6sYPvwb9u1LpFixEEaM6MDIkR2JjCzm7/CMMV7KKVFUcq94WsOpBHFSzrdz+1qoJYpgkJKSzrPP/sS+fYl07lyTSZMuo2FDa/XXmGCTU6IIBUrhXXPhBSvEjkYDVVJSCidOpFGmTATh4aFMmdKXLVsOcsstzeyeCGOCVE6JYqeqPlFgkeRFqCWKQDR37mYGD/6SLl1q8sYbVwLQsWNNOna09pmMCWY5JYrAPfwLi/R3BMbDzp1HuOeeuXz44VoASpYsRmJiCiVKWEI3pjDI6cL1bgUWRV6F2oNqAkFaWjrjxy8hJmYCH364lsjIMEaP7s7y5XdYkjCmEMm2RKGqBwoykDyxEoXfJSen0qnTWyxd6jy0sG/f+rz6am9q1Srr38CMMfkuz40CBoQwK1H4W0REGE2aVGLnzqO88kovrroqxk5WG1NIBWeisKueCpyq8tln66lcuRQXX+w8y2rMmJ6Ehoo1A25MIRekiSI4ww5Wf/55kKFDv+LLLzcRE1OBuLiBFC8eRtmyVrIzpigIzj2uJYoCceJEGi+99DNPPrmApKRUypQpzl13tSEszBrvM6YoCc49rlU9+dzChX8RGzuHdeucR4/cdNM/eOmlSznvvFJ+jswYU9CCNFEEZ9jBIikphWuv/Zg9e45Rt255Jk7sQ48eF/g7LGOMnwTnHleCM+xApqqkpSlhYSFERhZjzJhL2bhxP//9b0ciImx7G1OUBecewJrwyFfr1u0lNnY2PXrU4ZFHOgNw881N/RyVMSZQBOdZSStR5IvExBRGjvyOZs0ms3DhNqZOXcHx46n+DssYE2CCc49rJYpz9tVXmxgy5Ev+/PMQAAMHtuDZZ7tRvHhw/iSMMb4TnHsFO5l91o4dO8GAATP55JN1ADRtWpnJky+jXbvqfo7MGBOognOPa1VPZ61EiWIcOJBEyZLFePzxLtx1V1u7L8IYk6Pg3ONa1VOeLFv2N2XLRlC3bnlEhKlTLyc0NIQaNcr4OzRjTBAIzkNJK1F4JSEhmWHDvqR169eJjZ2NqvNgwtq1y1mSMMZ4LTj3uFaiyJGq8tFHa7n77rns2nWU0FDhoovOJzU1nWLFQv0dnjEmyARnorASRbb++OMAQ4Z8ydy5fwDQrl01Jk/uS9Omlf0cmTEmWAXnHteuesrSkSPHadnydQ4dSqZs2QhGj+7O7bdfREiIPSfCGHP2fLrHFZFewDggFJiqqs9lGn8zMMLtPQoMUtWVuS7YGgXMUlRUce65py2bNx/gxRcvpVKlkv4OyRhTCPgsUYhIKDAB6AHEA0tFZJaqrvOY7E+gs6oeFJHewBSgTa4LtxIFAHv3HuP++7+lW7fa9O/fDIBHHulkT5ozxuQrX1711BrYrKpbVPUEMB240nMCVf1ZVQ+6vb8C1bxachFPFOnpytSpv9GgwXjefnslDz30PSkpaQCWJIwx+c6XiaIqsN2jP94dlp3bgK+yGiEid4jIMhFZBhTpqqc1a/bQqdNb/Oc/X3DwYDLdu9fhu+9usauZjDE+48tD86wObTXLCUUuwUkUF2c1XlWn4FRL0bK6aFEsUSQlpTBq1I+MGfMrqanpVK5ckrFje3LDDU2sFGGM8Slf7nHjAc8GhKoBf2eeSESaAlOB3qq636slF8FEERIizJq1kbS0dAYPbsnTT3ezZ1YbYwqEL/e4S4F6IlIb2AHcANzkOYGI1AA+A/qr6kavlyzBeUN5XsXHH6ZEiWKULx9J8eJhTJvmnOJp08a7UznGGJMffLbHVdVUYCgwF1gPfKSqa0UkVkRi3ckeBaKBiSISl3EOoohLTU1n7NhfaNhwAvff/03G8DZtqlmSMMYUOJ/W4ajql8CXmYZN9ui+Hbg970suvHXyixfHM3DgbFau3A1AQsJxUlPTrYVXY4zfBGdlfyE8eXvoUDIjR37H5MnLUIWaNcswfnwf+vat7+/QjDFFXHAmikLm4MEkGjWayK5dRwkLC2H48HY88kgnSpYM93doxhgTrImicJUoypWLpHfvumzcuJ9Jky7jH/+wBvyMMYEjOBNFkFc9HT+eyujRi+jcuSadO9cCYPz4PkREhFkDfsaYgBOciSKISxTff/8ngwbNYePG/TRsWIHVqwcRGhpCiRJF925zY0xgC9JEEXz27DnG8OHf8O67qwCIianAxImXERpqVzMZYwJbcCaKIKp6OtmA34gR8zh0KJmIiDAefrgj99/fgfBwa5/JGBP4gjNRBFHVU0JCMg899D2HDiXTs+cFTJjQhwsuKO/vsIwxxmtBmigC27FjJwgLC6F48TDKlYtk8uTLSEtTrruukTXgZ4wJOkFaQR64O9tZs36nUaOJPP/8ooxh11zTiOuvb2xJwhgTlIIzUQTgDnfbtgSuumo6V145nW3bEpg79w/S07NsVd0YY4JKcCaKAJKSksaLL/5Mw4YTmDnzd6Kiwhk3rhfz5w+weyKMMYVCkJ6jCIwd8L59iXTr9j9WrXIa8LvuukaMHduTqlVL+zkyY4zJP8GZKAKk6ik6OpIKFUpQu3ZZxo/vQ58+9fwdkgkgKSkpxMfHk5yc7O9QTBESERFBtWrVKFYs/27iDc5E4SeqynvvraZ166rUrx+NiPDuu1dTpkyE3VltzhAfH09UVBS1atWyCxlMgVBV9u/fT3x8PLVr18635QbpOYqC/9P9/vs+und/h/79P2fw4DmoOieqzz8/ypKEyVJycjLR0dGWJEyBERGio6PzvRQbnCWKAvzjJSen8uyzC3nuuUWcOJFGdHQk//xn0wJbvwluliRMQfPFby44E0UBmTdvC4MGzWHz5gMA/PvfzXn++R5ER5fwc2TGGFNwrOopG7t3H6Vv3/fZvPkAjRpVZMGCAbzxxpWWJExQCQ0NpXnz5jRp0oTLL7+cQ4cOZYxbu3YtXbt2pX79+tSrV48nn3wyo0oV4KuvvqJly5Y0bNiQmJgY7rvvPj98gpytWLGC228/i6cpF5Djx4/Tr18/6tatS5s2bdi6dWuW03344Yc0bdqUxo0b88ADD5w27qOPPqJRo0Y0btyYm266CYC9e/fSq1cvX4d/iqoG1atFNVS3z1dfSEtL1/T09Iz+0aN/0mefXajHj6f6ZH2mcFu3bp2/Q9CSJUtmdN9yyy361FNPqapqYmKi1qlTR+fOnauqqseOHdNevXrp+PHjVVV19erVWqdOHV2/fr2qqqakpOiECRPyNbaUlJRzXsa1116rcXFxBbrOvJgwYYIOHDhQVVU/+OADvf7668+YZt++fVq9enXds2ePqjrf07x581RVdePGjdq8eXM9cOCAqqru3r07Y74BAwboTz/9lOV6s/rtAcv0LPe7QVr1lP8liri4XcTGzmbIkFb0798MgAce6JDv6zFF1Es+KgUP9/7u/3bt2rFqldPM/fvvv0+HDh249NJLAShRogTjx4+nS5cuDBkyhOeff56HHnqImJgYAMLCwhg8ePAZyzx69CjDhg1j2bJliAiPPfYY11xzDaVKleLo0aMAfPLJJ8yePZtp06YxYMAAypcvz4oVK2jevDmff/45cXFxlC1bFoC6deuyaNEiQkJCiI2NZdu2bQC8/PLLdOhw+v/xyJEjrFq1imbNnP/rkiVLuPvuu0lKSiIyMpK33nqLBg0aMG3aNObMmUNycjLHjh3jiy++YNiwYaxevZrU1FRGjRrFlVdeydatW+nfvz/Hjh0DYPz48bRv397r7ZuVmTNnMmrUKACuvfZahg4diqqedh5hy5Yt1K9fn4oVKwLQvXt3Pv30U7p168brr7/OkCFDKFeuHACVKlXKmO+qq67ivffeO2O7+EKQJor8c+TIcR577EfGjVtMerpy/Hga//xnUzsJaQqVtLQ0vvvuO2677TbAqXZq0aLFadNccMEFHD16lMOHD7NmzRqGDx+e63KffPJJypQpw+rVqwE4ePBgrvNs3LiRefPmERoaSnp6Op9//jm33norixcvplatWlSuXJmbbrqJe+65h4svvpht27bRs2dP1q9ff9pyli1bRpMmTTL6Y2JiWLBgAWFhYcybN4+RI0fy6aefAvDLL7+watUqypcvz8iRI+natStvvvkmhw4donXr1nTv3p1KlSrx7bffEhERwaZNm7jxxhtZtmzZGfF37NiRI0eOnDH8xRdfpHv37qcN27FjB9WrVwecZFumTBn2799PhQoVMqapW7cuGzZsYOvWrVSrVo0ZM2Zw4sSJjG0F0KFDB9LS0hg1alRGlVPLli15+OGHc93e+SFIE8W578RVlRkzNnDnnV8TH3+YkBDhrrva8MQTl1iSMPkvD0f++SkpKYnmzZuzdetWWrRoQY8ePQDOOKr1lJff/7x585g+fXpG/8kj35xcd911hIY6z2Lp168fTzzxBLfeeivTp0+nX79+Gctdt25dxjyHDx/myJEjREVFZQzbuXNnxlE4QEJCAv/617/YtGkTIkJKSkrGuB49elC+vNO8/zfffMOsWbN48cUXAecy5m3btlGlShWGDh1KXFwcoaGhGTvpzBYuXJjrZzxJ9czvPfP2LVeuHJMmTaJfv36EhITQvn17tmzZAkBqaiqbNm3ixx9/JD4+no4dO7JmzRrKli1LpUqV+Pvvv72O5VwEZ6I4xx35vn2J3HrrTGbPdn4ILVtW4bXX+nLRRefnR3TGBIzIyEji4uJISEigb9++TJgwgTvvvJPGjRuzYMGC06bdsmULpUqVIioqisaNG7N8+fKMap3sZJdwPIdlvqa/ZMmSGd3t2rVj8+bN7N27lxkzZmQcIaenp/PLL78QGRmZ42fzXPYjjzzCJZdcwueff87WrVvp0qVLlutUVT799FMaNGhw2vJGjRpF5cqVWblyJenp6URERGS53ryUKKpVq8b27dupVq0aqampJCQkZCQsT5dffjmXX345AFOmTMlIpNWqVaNt27YUK1aM2rVr06BBAzZt2kSrVq1ITk7OcfvkpyC96uncREWFs3nzAUqXLs748b359dfbLEmYQq1MmTK88sorvPjii6SkpHDzzTfz008/MW/ePMApedx5550ZV9zcf//9PPPMMxlH1enp6YwZM+aM5V566aWMHz8+o/9k1VPlypVZv359RtVSdkSEq6++mnvvvZeGDRsSHR2d5XLj4uLOmLdhw4Zs3rw5oz8hIYGqVasCMG3atGzX2bNnT1599dWMo/0VK1ZkzH/++ecTEhLCO++8Q1paWpbzL1y4kLi4uDNemZMEwBVXXMHbb78NOOdqunbtmmVi3bNnD+Bsv4kTJ2ZcyXXVVVfxww8/ALBv3z42btxInTp1AKdayrPqzZeCNFHkvUSxaNE29u9PBKB48TCmT7+GDRuGMGRIa3tutSkSLrzwQpo1a8b06dOJjIxk5syZPPXUUzRo0IB//OMftGrViqFDhwLQtGlTXn75ZW688UYaNmxIkyZN2Llz5xnLfPjhhzl48CBNmjShWbNmGTu15557jr59+9K1a1fOPz/ng7B+/frx7rvvZlQ7AbzyyissW7aMpk2b0qhRIyZPnnzGfDExMSQkJGQc3T/wwAP897//zajPz84jjzxCSkoKTZs2pUmTJjzyyCMADB48mLfffpu2bduycePG00ohZ+u2225j//791K1blzFjxvDcc89ljGvevHlG91133UWjRo3o0KEDDz74IPXr1wecpBYdHU2jRo245JJLeOGFFzKS6Q8//MBll112zjF6Q7KqQwtkLauLLlv8M1Rp59X0+/cn8uCD85g6dQW33XYhU6de4eMIjXGsX7+ehg0b+juMQm3s2LFERUUF9L0UvtKpUydmzpyZ5XmhrH57IrJcVVuezboK7aG0qvL223HExExg6tQVFCsWQpUqUVmeXDLGBKdBgwZRvHhxf4dR4Pbu3cu9997r1cUD+SE4T2bnUvW0YcM+YmNnM3/+XwB06VKLSZMuIyamQo7zGWOCS0REBP379/d3GAWuYsWKXHXVVQW2vuBMFDlc9RQff5hmzSZz4kQaFSqU4KWXLqV/f7svwvhHTpehGuMLvqg1Cc5EkYNq1UrTv39TQkKE557rTvnyBXP5mDGZRUREsH//fmtq3BQYdZ9Hkd2lvWcrSBPFqT/dzp1HuOeeucTGtqRLl1oATJlyuT2v2vhdtWrViI+PZ+/evf4OxRQhJ59wl5+CM1GIkJaWzqRJy3jooe85fPg4mzcfYOnS/yAiliRMQDh5k5Qxwc6nVz2JSC8R+V1ENovIg1mMFxF5xR2/SkQu8ma5v61KoG3bNxg27CsOHz7O5ZfX59NPr7fivTHG+IDPShQiEgpMAHoA8cBSEZmlqus8JusN1HNfbYBJ7nu2th8qTavei0hPd85HvPpqb668soElCWOM8RFflihaA5tVdYuqngCmA1dmmuZK4H9uc+m/AmVFJMfbOA8kRiIi3HtvW9avH8JVV8VYkjDGGB/y5TmKqsB2j/54ziwtZDVNVeC0tgJE5A7gDrf3ODy2ZswYyKLpmaKmArDP30EECNsWp9i2OMW2xSkNcp8ka75MFFkd5me+wNebaVDVKcAUABFZdra3oRc2ti1OsW1xim2LU2xbnCIiZz5cw0u+rHqKB6p79FcDMjee7s00xhhj/MiXiWIpUE9EaotIOHADMCvTNLOAW9yrn9oCCap6ZhOVxhhj/MZnVU+qmioiQ4G5QCjwpqquFZFYd/xk4EugD7AZSARu9WLRU3wUcjCybXGKbYtTbFucYtvilLPeFkHXzLgxxpiCVWibGTfGGJM/LFEYY4zJUcAmCl81/xGMvNgWN7vbYJWI/CwizfwRZ0HIbVt4TNdKRNJE5NqCjK8gebMtRKSLiMSJyFoRmV/QMRYUL/4jZUTkCxFZ6W4Lb86HBh0ReVNE9ojImmzGn91+U1UD7oVz8vsPoA4QDqwEGmWapg/wFc69GG2Bxf6O24/boj1Qzu3uXZS3hcd03+NcLHGtv+P24++iLLAOqOH2V/J33H7cFiOB0W53ReAAEO7v2H2wLToBFwFrshl/VvvNQC1R+KT5jyCV67ZQ1Z9V9aDb+yvO/SiFkTe/C4BhwKfAnoIMroB5sy1uAj5T1W0AqlpYt4c320KBKHHa+ymFkyhSCzZM31PVBTifLTtntd8M1ESRXdMeeZ2mMMjr57wN54ihMMp1W4hIVeBqYHIBxuUP3vwu6gPlRORHEVkuIrcUWHQFy5ttMR5oiHND72rgLlVNL5jwAspZ7TcD9XkU+db8RyHg9ecUkUtwEsXFPo3If7zZFi8DI1Q1rZA3FunNtggDWgDdgEjgFxH5VVU3+jq4AubNtugJxAFdgQuAb0Vkoaoe9nFsgeas9puBmiis+Y9TvPqcItIUmAr0VtX9BRRbQfNmW7QEprtJogLQR0RSVXVGgURYcLz9j+xT1WPAMRFZADQDClui8GZb3Ao8p05F/WYR+ROIAZYUTIgB46z2m4Fa9WTNf5yS67YQkRrAZ0D/Qni06CnXbaGqtVW1lqrWAj4BBhfCJAHe/UdmAh1FJExESuC03ry+gOMsCN5si204JStEpDJOS6pbCjTKwHBW+82ALFGo75r/CDpebotHgWhgonsknaqFsMVML7dFkeDNtlDV9SLyNbAKSAemqmqWl00GMy9/F08C00RkNU71ywhVLXTNj4vIB0AXoIKIxAOPAcXg3Pab1oSHMcaYHAVq1ZMxxpgAYYnCGGNMjixRGGOMyZElCmOMMTmyRGGMMSZHlihMQHJbfo3zeNXKYdqj+bC+aSLyp7uu30Sk3VksY6qINHK7R2Ya9/O5xugu5+R2WeO2hlo2l+mbi0if/Fi3Kbrs8lgTkETkqKqWyu9pc1jGNGC2qn4iIpcCL6pq03NY3jnHlNtyReRtYKOqPp3D9AOAlqo6NL9jMUWHlShMUBCRUiLynXu0v1pEzmg1VkTOF5EFHkfcHd3hl4rIL+68H4tIbjvwBUBdd9573WWtEZG73WElRWSO+2yDNSLSzx3+o4i0FJHngEg3jvfccUfd9w89j/Ddksw1IhIqIi+IyFJxnhMw0IvN8gtug24i0lqcZ5GscN8buHcpPwH0c2Pp58b+prueFVltR2PO4O/20+1lr6xeQBpOI25xwOc4rQiUdsdVwLmz9GSJ+Kj7Phx4yO0OBaLcaRcAJd3hI4BHs1jfNNxnVwDXAYtxGtRbDZTEaZp6LXAhcA3wuse8Zdz3H3GO3jNi8pjmZIxXA2+73eE4LXlGAncAD7vDiwPLgNpZxHnU4/N9DPRy+0sDYW53d+BTt3sAMN5j/meAf7rdZXHafSrp7+/bXoH9CsgmPIwBklS1+ckeESkGPCMinXCao6gKVAZ2ecyzFHjTnXaGqsaJSGegEbDIbd4kHOdIPCsviMjDwF6cVni7AZ+r06geIvIZ0BH4GnhRREbjVFctzMPn+gp4RUSKA72ABaqa5FZ3NZVTT+QrA9QD/sw0f6SIxAG1gOXAtx7Tvy0i9XBaAy2WzfovBa4Qkfvc/gigBoWzDSiTTyxRmGBxM86TyVqoaoqIbMXZyWVQ1QVuIrkMeEdEXgAOAt+q6o1erON+Vf3kZI+IdM9qIlXdKCItcNrMeVZEvlHVJ7z5EKqaLCI/4jR73Q/44OTqgGGqOjeXRSSpanMRKQPMBoYAr+C0ZfSDql7tnvj/MZv5BbhGVX/3Jl5jwM5RmOBRBtjjJolLgJqZJxCRmu40rwNv4DwS8legg4icPOdQQkTqe7nOBcBV7jwlcaqNFopIFSBRVd8FXnTXk1mKW7LJynScxtg64jRkh/s+6OQ8IlLfXWeWVDUBuBO4z52nDLDDHT3AY9IjOFVwJ80FholbvBKRC7NbhzEnWaIwweI9oKWILMMpXWzIYpouQJyIrMA5jzBOVffi7Dg/EJFVOIkjxpsVqupvOOculuCcs5iqqiuAfwBL3Cqgh4Cnsph9CrDq5MnsTL7BebbxPHUe3QnOs0TWAb+JyBrgNXIp8buxrMRpVvt5nNLNIpzzFyf9ADQ6eTIbp+RRzI1tjdtvTI7s8lhjjDE5shKFMcaYHFmiMMYYkyNLFMYYY3JkicIYY0yOLFEYY4zJkSUKY4wxObJEYYwxJkf/D/MYAKWWjFT1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_curve, auc\n",
    "fpr = dict()\n",
    "tpr = dict()\n",
    "roc_auc = dict()\n",
    "for i in range(2):   #两类\n",
    "    #计算每个target的fpr和tpr\n",
    "    fpr[i], tpr[i], _ = roc_curve(y_test[:, i], y_score[:, i])  #y_test \n",
    "    roc_auc[i] = auc(fpr[i], tpr[i])  # auc直接计算面积\n",
    "    \n",
    "# Compute micro-average ROC curve and ROC area\n",
    "fpr[\"micro\"], tpr[\"micro\"], _ = roc_curve(y_test.ravel(), y_score.ravel())\n",
    "roc_auc[\"micro\"] = auc(fpr[\"micro\"], tpr[\"micro\"])\n",
    "\n",
    "plt.figure()\n",
    "id=1  # 选择不同的id即绘制不同target分类的效果\n",
    "lw = 2\n",
    "plt.plot(fpr[id], tpr[id], color='darkorange',\n",
    "         lw=lw, label='ROC curve (area = %0.2f)' % roc_auc[1])#S1=0.97\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver operating characteristic example')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;综合上述情况，随机森林的预测精度更高，模型稳定性更好，神经网络与随机森林的AUC指数均为0.96，两种分类模型都具有较好的实用性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PCA降维操作尝试优化模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;针对模型的改进，本人的思路来源于对数据的直观观察。在观察数据时发现：虽然输入的数据维数很高，但大多数都是0，因此，本人通过数据降维的方法对数据信息进行提取，发现在数据维度大大降低之后，原数据的信息量并没有过多丢失。应用到实际情况，可以大大降低考察人员的工作负担，让工作效率大大提高。<br>\n",
    "&ensp;&ensp;&ensp;&ensp;本人在进行数据降维时，采用的是pca降维法，pca降维可以直接按照信息量大小来进行降维操作，避免了无目的性的降维。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 将pca降维后的数据代入神经网络和随机森林中<br>\n",
    "**首先代入到神经网络中：**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "G:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\validation.py:72: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(**kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.91271\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "num,q1,q2,q3,q4,q5,q6,q7,q8,q9,q10,q11,q12,q13,q14,q15,q16,q17,q18,q19,q20,q21,q22,q23,q24,q25,q26,q27,q28,q29,q30,q31,q32,q33,q34,q35,q36,y= np.loadtxt(\"train.csv\",delimiter=',', usecols=(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37), \n",
    "\tunpack=True)\n",
    "num = num.reshape(-1,1)\n",
    "y = y.reshape(-1,1)\n",
    "for i in range(1,37):\n",
    "    exec (\"q%s=q%s.reshape(-1,1)\"%(i,i))\n",
    "train = np.hstack((q1.reshape(-1,1),q2.reshape(-1,1)))\n",
    "for i in range(3,37):\n",
    "    exec (\"train=np.hstack((train,q%s.reshape(-1,1)))\"%i)\n",
    "#产生训练集\n",
    "\n",
    "#提取主要信息，进行降维处理\n",
    "from sklearn.decomposition import PCA\n",
    "pca = PCA(n_components=0.95)#降维到5维\n",
    "proj = pca.fit_transform(train)\n",
    "\n",
    "#利用降维后的数据进行分类\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "clf_class= MLPClassifier(solver='adam', alpha=1e-5,hidden_layer_sizes=(40,20), random_state=1,max_iter=2000)\n",
    "clf_class.fit(proj,y)\n",
    "#sklearn神经网络学习，隐藏层两层，第一层40个节点，第二层20个节点，最多迭代2000次\n",
    "\n",
    "y_pred=[]\n",
    "j=0\n",
    "for i in range(np.shape(train)[0]):\n",
    "    y_pred.append(clf_class.predict([proj[i]]))\n",
    "for i in range(np.shape(train)[0]):\n",
    "    if y_pred[i]==y[i]:\n",
    "        j=j+1\n",
    "print(j/200000)\n",
    "#未经过pca降维处理的训练集准确率：0.925\n",
    "#经过pca降维处理后的训练集准确率：0.913\n",
    "#相同层数的隐藏层，准确率下降，计算时间明显延长"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**再将其代入到随机森林分类模型中：**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "num,q1,q2,q3,q4,q5,q6,q7,q8,q9,q10,q11,q12,q13,q14,q15,q16,q17,q18,q19,q20,q21,q22,q23,q24,q25,q26,q27,q28,q29,q30,q31,q32,q33,q34,q35,q36,y= np.loadtxt(\"train.csv\",delimiter=',', usecols=(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37), \n",
    "\tunpack=True)\n",
    "num = num.reshape(-1,1)\n",
    "y = y.reshape(-1,1)\n",
    "for i in range(1,37):\n",
    "    exec (\"q%s=q%s.reshape(-1,1)\"%(i,i))\n",
    "train = np.hstack((q1.reshape(-1,1),q2.reshape(-1,1)))\n",
    "for i in range(3,37):\n",
    "    exec (\"train=np.hstack((train,q%s.reshape(-1,1)))\"%i)\n",
    "# 产生训练集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "#提取主要信息，进行降维处理\n",
    "from sklearn.decomposition import PCA\n",
    "pca = PCA(n_components=0.95)#降维到5维\n",
    "proj = pca.fit_transform(train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-21-ac05cbc07a0c>:5: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().\n",
      "  rfc = rfc.fit(proj,y)\n"
     ]
    }
   ],
   "source": [
    "#利用降维后的数据进行分类\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rfc = RandomForestClassifier(n_estimators=10, random_state=0)\n",
    "#第一个参数代表随机森林中树木的个数，往往是越多越好，但准确率会收敛，参数越大，计算时间越长\n",
    "rfc = rfc.fit(proj,y)       \n",
    "#用训练集数据训练模型 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.931045\n"
     ]
    }
   ],
   "source": [
    "y_pred=[]\n",
    "j=0\n",
    "for i in range(np.shape(train)[0]):\n",
    "    y_pred.append(rfc.predict([proj[i]]))\n",
    "for i in range(np.shape(train)[0]):\n",
    "    if y_pred[i]==y[i]:\n",
    "        j=j+1\n",
    "print(j/200000)\n",
    "#未经过pca降维处理的准确率：0.931\n",
    "#经过pca降维处理后的准确率：0.931，准确率没有变化，收敛速度也没有变化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp;&ensp;&ensp;&ensp;综上所述，pca降维操作的确可以大大简化数据的复杂程度，但是却将神经模型的收敛速度延长了，而对随机森林模型没有明显影响，由此可见，随机森林模型准确率高的同时不受数据维度影响，模型稳定性强；pca降维操作并不能将模型优化，但却可以给实际生活带来重大意义：**降低事故现场分析员的工作负担，提高工作效率，精简工作流程**。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
